Re: GCC 4.1: Buildable on GHz machines only?
On Tue, Apr 26, 2005 at 07:50:40PM -0700, Matt Thomas wrote: > Over the past month I've been making sure that GCC 4.1 works on NetBSD. > I've completed bootstraps on sparc, sparc64, arm, x86_64, i386, alpha, > mipsel, mipseb, and powerpc. I've done cross-build targets for vax. > Results have been sent to gcc-testsuite. > > The times to complete bootstraps on older machines has been bothering me. > It took nearly 72 hours for 233MHz StrongArm with 64MB to complete a > bootstrap (with libjava). It took over 48 hours for a 120MHz MIPS R4400 > (little endian) with 128MB to finish (without libjava) and a bit over 24 > hours for a 250MHz MIPS R4400 (big endian) with 256MB to finish (again, > no libjava). That doesn't even include the time to run the testsuites. > > I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours > (48 hours just to exit stage3 and start on the libraries) doing a bootstrap > knowing that it's going to die when doing the ranlib of libjava. The kernel > for the 060 isn't configured with a large enough dataspace to complete the > ranlib. > > Most of the machines I've listed above are relatively powerful machines > near the apex of performance of their target architecture. And yet GCC4.1 > can barely be bootstrapped on them. Note that the MIPSen are not near the top of modern MIPS performance. The ARM isn't quite there either, but the higher-powered ARMs are a bit scarcer than the MIPS. None of this detracts from your point, though. > I'm going to run some bootstraps with --disable-checking just to see how > much faster they are. I hope I'm going to pleasantly surprised but I'm > not counting on it. I would expect it to be drastically faster. However this won't show up clearly in the bootstrap. The, bar none, longest bit of the bootstrap is building stage2; and stage1 is always built with optimization off and (IIRC) checking on. -- Daniel Jacobowitz CodeSourcery, LLC
Re: GCC 4.1: Buildable on GHz machines only?
On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote: > I would expect it to be drastically faster. However this won't show up > clearly in the bootstrap. The, bar none, longest bit of the bootstrap > is building stage2; and stage1 is always built with optimization off and > (IIRC) checking on. Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when building on risc machines. r~
Re: GCC 4.1: Buildable on GHz machines only?
Richard Henderson wrote: > On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote: > >>I would expect it to be drastically faster. However this won't show up >>clearly in the bootstrap. The, bar none, longest bit of the bootstrap >>is building stage2; and stage1 is always built with optimization off and >>(IIRC) checking on. > > > Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when > building on risc machines. Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was already doing) only decreased the bootstrap time by 10%. By far, the longest bit of the bootstrap is building libjava. -- Matt Thomas email: [EMAIL PROTECTED] 3am Software Foundry www: http://3am-software.com/bio/matt/ Cupertino, CA disclaimer: I avow all knowledge of this message.
RE: GCC 4.1: Buildable on GHz machines only?
> -Original Message- > From: Matt Thomas > Sent: Tuesday, April 26, 2005 10:42 PM [...] > > Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was > already doing) only decreased the bootstrap time by 10%. By far, the > longest bit of the bootstrap is building libjava. > Is it fair to compare current build times, with libjava included, against past build times when it didn't exist? Would a closer apples-to-apples comparison be to bootstrap GCC Core only on the older sub Ghz platforms?
Re: GCC 4.1: Buildable on GHz machines only?
Matt Thomas <[EMAIL PROTECTED]> writes: > I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours > (48 hours just to exit stage3 and start on the libraries) doing a bootstrap > knowing that it's going to die when doing the ranlib of libjava. The kernel > for the 060 isn't configured with a large enough dataspace to complete the > ranlib. If so, that is particularly irritating since the ranlib is completely unnecessary. GNU ar always builds an archive symbol table by default. Ian
Re: GCC 4.1: Buildable on GHz machines only?
Gary Funck wrote: > >>-Original Message- >>From: Matt Thomas >>Sent: Tuesday, April 26, 2005 10:42 PM > > [...] > >>Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was >>already doing) only decreased the bootstrap time by 10%. By far, the >>longest bit of the bootstrap is building libjava. >> > > > Is it fair to compare current build times, with libjava included, > against past build times when it didn't exist? Would a closer > apples-to-apples comparison be to bootstrap GCC Core only on > the older sub Ghz platforms? libjava is built on everything but vax and mips. Bootstrapping core might be better but do the configure on the fly it's not as easy as it used to be. It would be nice if bootstrap emitted timestamps when it was started and when it completed a stage so one could just look at the make output. Regardless, GCC4.1 is a computational pig. -- Matt Thomas email: [EMAIL PROTECTED] 3am Software Foundry www: http://3am-software.com/bio/matt/ Cupertino, CA disclaimer: I avow all knowledge of this message.
Re: GCC 4.1: Buildable on GHz machines only?
Matt Thomas <[EMAIL PROTECTED]> writes: > libjava is built on everything but vax and mips. Bootstrapping core > might be better but do the configure on the fly it's not as easy as > it used to be. --enable-languages=c,c++ (or even perhaps --enable-languages=c) doesn't work for you? Also, I believe "make all-gcc TARGET-gcc=bootstrap" will bootstrap the compiler normally but not build the runtime libraries. > It would be nice if bootstrap emitted timestamps when it was started > and when it completed a stage so one could just look at the make output. Patches are welcome. zw
Re: GCC 4.1: Buildable on GHz machines only?
Matt Thomas <[EMAIL PROTECTED]> writes: > Richard Henderson wrote: > > On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote: > > > >>I would expect it to be drastically faster. However this won't show up > >>clearly in the bootstrap. The, bar none, longest bit of the bootstrap > >>is building stage2; and stage1 is always built with optimization off and > >>(IIRC) checking on. > > > > > > Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when > > building on risc machines. > > Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was I don't think that is enough, also edit gcc/Makefile.in and change the line: STAGE1_CHECKING = -DENABLE_CHECKING -DENABLE_ASSERT_CHECKING to be STAGE1_CHECKING = Is there a better way to do this? STAGE1_CHECKING is not passed from the toplevel make, so one cannot put it on the make bootstrap command line...
Re: GCC 4.1: Buildable on GHz machines only?
> Matt Thomas writes: Matt> Regardless, GCC4.1 is a computational pig. If you are referring to the compiler itself, this has no basis in reality. If you are referring to the entire compiler collection, including runtimes, you are not using a fair comparison or are making extreme statements without considering the cause. GCC now supports C++, Fortran 90 and Java. Those languages have extensive, complicated runtimes. The GCC Java environment is becoming much more complete and standards compliant, which means adding more and more features. If your point is that fully supporting modern, richly featured languages results in a longer build process, that is correct. Using disparaging terms like "pig" is missing the point. As others have pointed out, if you do not want to build some languages and runtimes, you can disable them. GCC is providing features that users want and that has a cost. David
Re: GCC 4.1: Buildable on GHz machines only?
David Edelsohn wrote: >>Matt Thomas writes: > > > Matt> Regardless, GCC4.1 is a computational pig. > > If you are referring to the compiler itself, this has no basis in > reality. If you are referring to the entire compiler collection, > including runtimes, you are not using a fair comparison or are making > extreme statements without considering the cause. When I see the native stage2 m68k compiler spend 30+ minutes compute bound with no paging activity compiling a single source file, I believe that is an accurate term. Compiling stage3 on a 50MHz 68060 took 18 hours. (That 30 minutes was for fold-const.c if you care to know). At some points, I had no idea whether GCC had just gone into an infinite loop due a bug or was actually doing what it was supposed to. > GCC now supports C++, Fortran 90 and Java. Those languages have > extensive, complicated runtimes. The GCC Java environment is becoming > much more complete and standards compliant, which means adding more and > more features. That's all positive but if GCC also becomes too expensive to build then all those extra features become worthless. What is the slowest system that GCC has been recently bootstrapped on? > If your point is that fully supporting modern, richly featured > languages results in a longer build process, that is correct. Using > disparaging terms like "pig" is missing the point. As others have pointed > out, if you do not want to build some languages and runtimes, you can > disable them. GCC is providing features that users want and that has a > cost. Yes they have a cost, but the cost is mitigated by running fast processors. They are just so fast they can hide ineffiences and bloat. We have seen that for NetBSD and it's just as true for GCC or any other software. These slower processor perform usefull feedback but only if a GCC bootstrap is attempted on them on a semi-regular basis. Am I the only person who has attempted to do a native bootstrap on a system as slow as a M68k? I thought about doing a bootstrap on a MicroSparc based system but instead I decided to use a UltraSparcIIi system running with a 32bit kernel. -- Matt Thomas email: [EMAIL PROTECTED] 3am Software Foundry www: http://3am-software.com/bio/matt/ Cupertino, CA disclaimer: I avow all knowledge of this message.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 08:05:39AM -0700, Matt Thomas wrote: > David Edelsohn wrote: > > > GCC now supports C++, Fortran 90 and Java. Those languages have > > extensive, complicated runtimes. The GCC Java environment is becoming > > much more complete and standards compliant, which means adding more and > > more features. > > That's all positive but if GCC also becomes too expensive to build then > all those extra features become worthless. Worthless to whom? The features under discussion are new, they didn't exist before. If you survived without them previously you can do so now. (i.e. don't build libjava if your machine isn't capable of it) But claiming it's "worthless" when plenty of people are using it is just, well ... worthless. jon -- "Speed, it seems to me, provides the one genuinely modern pleasure." - Aldous Huxley
Re: GCC 4.1: Buildable on GHz machines only?
> Matt Thomas writes: Matt> That's all positive but if GCC also becomes too expensive to build then Matt> all those extra features become worthless. What is the slowest system Matt> that GCC has been recently bootstrapped on? GCC recently was bootstrapped on a VAX. The GCC build times are not unreasonable compared to other, commercial compilers with similar functionality. And the GCC developers ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC 3.4. David
Re: GCC 4.1: Buildable on GHz machines only?
Jonathan Wakely wrote: > On Wed, Apr 27, 2005 at 08:05:39AM -0700, Matt Thomas wrote: > > >>David Edelsohn wrote: >> >> >>> GCC now supports C++, Fortran 90 and Java. Those languages have >>>extensive, complicated runtimes. The GCC Java environment is becoming >>>much more complete and standards compliant, which means adding more and >>>more features. >> >>That's all positive but if GCC also becomes too expensive to build then >>all those extra features become worthless. > > > Worthless to whom? To users of that platform that can no longer afford to build GCC. > The features under discussion are new, they didn't exist before. And because they never existed before, their cost for older platforms may not have been correctly assessed. If no one builds natively on older platforms, the recognition that the new features maybe a problem for older platforms will never be made. > If you survived without them previously you can do so now. > (i.e. don't build libjava if your machine isn't capable of it) Yes, you can skip building libjava. But can you skip building GCC? Will GCC 3.x be supported forever? If not, your compiler may have to rely being cross-built. Being able to do a bootstrap is useful and is part of the expected GCC testing but when it can only be done one or two a week, it becomes a less practical test method. > But claiming it's "worthless" when plenty of people are using it is > just, well ... worthless. Depends on your point of view. -- Matt Thomas email: [EMAIL PROTECTED] 3am Software Foundry www: http://3am-software.com/bio/matt/ Cupertino, CA disclaimer: I avow all knowledge of this message.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 16:31, David Edelsohn wrote: > The GCC build times are not unreasonable compared to other, > commercial compilers with similar functionality. And the GCC developers > ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC > 3.4. If you are going to make sweeping statements like this you need to back them up with hard data. For C code, on average (obviously there are cases which are exceptions) GCC 4.0 is about 7% slower on ARM than gcc 3.4. http://www.inf.u-szeged.hu/csibe/ocomp.php?branchid_a=gcc_3_4_0_release&branchid_b=gcc_4_0_0_release&targetid_a=arm-elf&targetid_b=arm-elf×tamp_a=2003-01-01%2012:00:00×tamp_b=2003-01-01%2012:00:00&flags_a=-O2&flags_b=-O2&csibever_a=2.x.x&csibever_b=2.x.x&dataview=Compilation%20time&viewmode=Summarized%20bar%20chart&finish_button=Finish And it is >10% slower than 3.3. And ... R.
RE: GCC 4.1: Buildable on GHz machines only?
Original Message >From: David Edelsohn >Sent: 27 April 2005 16:32 >> Matt Thomas writes: > > Matt> That's all positive but if GCC also becomes too expensive to build > then Matt> all those extra features become worthless. What is the > slowest system Matt> that GCC has been recently bootstrapped on? > > GCC recently was bootstrapped on a VAX. > > The GCC build times are not unreasonable compared to other, > commercial compilers with similar functionality. And the GCC developers > ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC > 3.4. > > David I'm a bit concerned about the increasing bootstrap times as well. I wanted to test a small patch at the weekend, so I checked out a couple of copies of HEAD, modified one slightly, then set two full "make bootstrap; make check" runs going. This is on a 850Mhz Athlon. It took over 24 hours. In 2.95.x days it would have taken maybe three or three-and-a-bit hours on the very same machine. This makes it very much slower and more difficult to submit properly tested patches. I don't have any good ideas how to deal with this really, without cutting down on the overall amount of testing (either explicitly, by not running the full testsuite, or implicitly, by disabling most of the languages at configure time). I would like to be thorough, but find it quite a handicap. Suggestions are welcome! cheers, DaveK -- Can't think of a witty .sigline today
Re: GCC 4.1: Buildable on GHz machines only?
> Richard Earnshaw writes: Richard> On Wed, 2005-04-27 at 16:31, David Edelsohn wrote: >> The GCC build times are not unreasonable compared to other, >> commercial compilers with similar functionality. And the GCC developers >> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC >> 3.4. Richard> If you are going to make sweeping statements like this you need to back Richard> them up with hard data. What does gcc_4_0_0_release branch at time 2003-01-01 mean? David
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 17:19, David Edelsohn wrote: > > Richard Earnshaw writes: > > Richard> On Wed, 2005-04-27 at 16:31, David Edelsohn wrote: > >> The GCC build times are not unreasonable compared to other, > >> commercial compilers with similar functionality. And the GCC developers > >> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC > >> 3.4. > > Richard> If you are going to make sweeping statements like this you need to > back > Richard> them up with hard data. > > What does gcc_4_0_0_release branch at time 2003-01-01 mean? Hmm, you'll have to ask the CSiBE folks that one... R.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 17:29, Richard Earnshaw wrote: > On Wed, 2005-04-27 at 17:19, David Edelsohn wrote: > > > Richard Earnshaw writes: > > > > Richard> On Wed, 2005-04-27 at 16:31, David Edelsohn wrote: > > >> The GCC build times are not unreasonable compared to other, > > >> commercial compilers with similar functionality. And the GCC developers > > >> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC > > >> 3.4. > > > > Richard> If you are going to make sweeping statements like this you need to > > back > > Richard> them up with hard data. > > > > What does gcc_4_0_0_release branch at time 2003-01-01 mean? > > Hmm, you'll have to ask the CSiBE folks that one... In fact the time is irrelevant in this case, since it's not a branch but a specific tag on a single version. R.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 08:45:01AM -0700, Matt Thomas wrote: > > If you survived without them previously you can do so now. > > (i.e. don't build libjava if your machine isn't capable of it) > > Yes, you can skip building libjava. But can you skip building GCC? > Will GCC 3.x be supported forever? If not, your compiler may have > to rely being cross-built. Being able to do a bootstrap is useful > and is part of the expected GCC testing but when it can only be > done one or two a week, it becomes a less practical test method. I have no idea what you're trying to say in this paragraph. Not only can you skip building libjava, you can skip building the compilers for any languages that you do not want to test. -- Daniel Jacobowitz CodeSourcery, LLC
Re: GCC 4.1: Buildable on GHz machines only?
Matt Thomas wrote: Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was already doing) only decreased the bootstrap time by 10%. By far, the longest bit of the bootstrap is building libjava. Building libjava takes forever on any platform, relative to the rest of the compiler build. That's why I often disable it, especially since I don't use Java very much, and gcj even less often than Java in general. Do you really *need* gcj on these embedded platforms? Also, I don't really think that bootstrapping GCC directly on target hardware is very useful -- except for testing GCC. I'd build GCC on some fast workstation, even if I eventually wanted it to run native on the target. -- Mark Mitchell CodeSourcery, LLC [EMAIL PROTECTED] (916) 791-8304
Re: GCC 4.1: Buildable on GHz machines only?
> "Ian" == Ian Lance Taylor writes: >> Matt Thomas <[EMAIL PROTECTED]> writes: >> I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours >> (48 hours just to exit stage3 and start on the libraries) doing a bootstrap >> knowing that it's going to die when doing the ranlib of libjava. The kernel >> for the 060 isn't configured with a large enough dataspace to complete the >> ranlib. Ian> If so, that is particularly irritating since the ranlib is completely Ian> unnecessary. GNU ar always builds an archive symbol table by default. Matt, could you file a PR for this? I agree with Ian, we really shouldn't be running ranlib here. Perhaps we can get the libtool maintainers to fix this. Tom
Re: GCC 4.1: Buildable on GHz machines only?
On Wednesday 27 April 2005 17:45, Matt Thomas wrote: > > The features under discussion are new, they didn't exist before. > > And because they never existed before, their cost for older platforms > may not have been correctly assessed. If someone had cared about them, it would have been noticed earlier. But since _nobody_ has complained before you, I guess we can conclude that by far the majority if GCC users are quite happy with the cost assesments that were made. > If no one builds natively on > older platforms, the recognition that the new features maybe a problem > for older platforms will never be made. Maybe the older platform should stick to the older compiler then, if it is too slow to support the kind of compiler that modern systems need. Gr. Steven
Re: GCC 4.1: Buildable on GHz machines only?
> "Steven" == Steven Bosscher <[EMAIL PROTECTED]> writes: Steven> On Wednesday 27 April 2005 17:45, Matt Thomas wrote: >> If no one builds natively on older platforms, the recognition that >> the new features maybe a problem for older platforms will never be >> made. Steven> Maybe the older platform should stick to the older compiler Steven> then, if it is too slow to support the kind of compiler that Steven> modern systems need. Maybe I'm missing something, but... Isn't a full bootstrap (all languages) part of the required test procedure for changes? That's what the website says right now. Since Matt is the Vax port maintainer, he therefore has good reasons for needing to run bootstraps on slow machines. Your comment seems to translate to: "when GCC grows to the point that you can't reasonably run a full bootstrap on platform X anymore, then platform X is obsolete". That seems like a strange new obsoletion criterion. The alternative of course is to do only crossbuilds. Is it reasonable to say that, for platforms where a bootstrap is no longer feasible, a successful crossbuild is an acceptable test procedure to use instead? paul
Re: GCC 4.1: Buildable on GHz machines only?
> The alternative of course is to do only crossbuilds. Is it reasonable > to say that, for platforms where a bootstrap is no longer feasible, a > successful crossbuild is an acceptable test procedure to use instead? > Sure, and get flamed and trounced by Uli on glibc when you talk about problems with crossbuilding. -Steve
Re: GCC 4.1: Buildable on GHz machines only?
On Wednesday 27 April 2005 22:06, Paul Koning wrote: > > "Steven" == Steven Bosscher <[EMAIL PROTECTED]> writes: > > Steven> On Wednesday 27 April 2005 17:45, Matt Thomas wrote: > >> If no one builds natively on older platforms, the recognition that > >> the new features maybe a problem for older platforms will never be > >> made. > > Steven> Maybe the older platform should stick to the older compiler > Steven> then, if it is too slow to support the kind of compiler that > Steven> modern systems need. > > Maybe I'm missing something, but... > > Isn't a full bootstrap (all languages) part of the required test > procedure for changes? That's what the website says right now. Isn't there a special text about port changes? Some ports don't even support all languages. > Since > Matt is the Vax port maintainer, he therefore has good reasons for > needing to run bootstraps on slow machines. Yes, he has. Is that a valid reason to call GCC4 a pig? No. > Your comment seems to translate to: "when GCC grows to the point that > you can't reasonably run a full bootstrap on platform X anymore, then > platform X is obsolete". That seems like a strange new obsoletion > criterion. Interesting way of arguing. First you twist my words and put something in my mouth that I did not say, then you comment on that... What I'm saying is that if you really want/need for some reason to do full bootstraps of the latest and greatest GCC on something as old and slow as m68k (the old kind), VAX, or PDP-11, you should not complain that other people have moved on to recent targets where a bootstrap is not such a big deal and the new features in GCC make the difference between a good or poor compiler for that target. I don't think there's any disagreement that GCC is not as fast as it should be. But bootstrapping [*] lines of code on a real VAX is just never going to be very fast. There is no reason to blame GCC for that. Gr. Steven [*] Does anyone have an idea of how large GCC really is?
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 10:36:08PM +0200, Steven Bosscher wrote: > [*] Does anyone have an idea of how large GCC really is? > ~1.8 MLOC. Courtesy of David Wheeler's SLOCCount (testsuites excluded): SLOCDirectory SLOC-by-Language (Sorted) 1179994 gcc ansic=745370,ada=395409,asm=21934,yacc=13865,sh=1841, lex=857,awk=439,perl=193,pascal=86 398689 libjava java=282613,cpp=69449,ansic=39675,sh=5417,perl=858, yacc=653,awk=24 50724 libstdc++-v3cpp=34477,ansic=15657,sh=421,perl=130,awk=39 31583 libgfortran ansic=31166,f90=365,sh=52 30400 boehm-gcansic=28510,cpp=1146,asm=443,sh=301 23021 libiberty ansic=22722,perl=299 19533 zlibansic=13100,asm=2710,ada=1637,pascal=1085,cpp=1001 13772 libffi ansic=8135,asm=5637 12324 libcpp ansic=12324 12102 top_dir sh=12102 8565libobjc ansic=7752,objc=427,cpp=386 6955intlansic=6639,yacc=316 4861libmudflap ansic=4861 3958contrib cpp=2306,sh=1122,perl=374,awk=67,lisp=59,ansic=30 2758fastjar ansic=2758 2697fixincludes ansic=2254,sh=443 1734include ansic=1706,cpp=28 837 maintainer-scripts sh=828,perl=9 Totals grouped by language (dominant language first): ansic: 942659 (52.24%) ada: 397046 (22.00%) java:282613 (15.66%) cpp: 108793 (6.03%) asm: 30724 (1.70%) sh: 22527 (1.25%) yacc: 14834 (0.82%) perl: 1863 (0.10%) pascal:1171 (0.06%) lex:857 (0.05%) awk:569 (0.03%) objc: 427 (0.02%) f90:365 (0.02%) lisp:59 (0.00%) Total Physical Source Lines of Code (SLOC)= 1,804,507 Development Effort Estimate, Person-Years (Person-Months) = 525.06 (6,300.68) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) Schedule Estimate, Years (Months) = 5.79 (69.46) (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) Estimated Average Number of Developers (Effort/Schedule) = 90.72 Total Estimated Cost to Develop = $ 70,928,067 (average salary = $56,286/year, overhead = 2.40). SLOCCount, Copyright (C) 2001-2004 David A. Wheeler SLOCCount is Open Source Software/Free Software, licensed under the GNU GPL. SLOCCount comes with ABSOLUTELY NO WARRANTY, and you are welcome to redistribute it under certain conditions as specified by the GNU GPL license; see the documentation for details. Please credit this data as "generated using David A. Wheeler's 'SLOCCount'."
Re: GCC 4.1: Buildable on GHz machines only?
> "Steven" == Steven Bosscher <[EMAIL PROTECTED]> writes: Steven> On Wednesday 27 April 2005 22:06, Paul Koning wrote: >> Isn't a full bootstrap (all languages) part of the required test >> procedure for changes? That's what the website says right now. Steven> Isn't there a special text about port changes? Some ports Steven> don't even support all languages. http://gcc.gnu.org/contribute.html#testing does not list any special text that affects the testing rules, at least not that I can find. Yes, I suppose not all ports don't support all languages. I don't know if that's true for VAX. If so, then I would assume the rule becomes "all supported languages". Steven> What I'm saying is that if you really want/need for some Steven> reason to do full bootstraps of the latest and greatest GCC Steven> on something as old and slow as m68k (the old kind), VAX, or Steven> PDP-11, you should not complain that other people have moved Steven> on to recent targets where a bootstrap is not such a big deal Steven> and the new features in GCC make the difference between a Steven> good or poor compiler for that target. So long as the testing rules are what they are, I can't agree with that. If the testing rules are changed to allow less testing when the platform in question is slow, that's a different matter. Note that the PDP-11 case is different, that's not a native target. Also, when platform obsoletion is discussed, lack of test results is often mentioned as a metric. It seems unfair to complain about lack of test results when the software being tested is the cause (or part of the cause), not necessarily the lack of tester interest. Steven> I don't think there's any disagreement that GCC is not as Steven> fast as it should be. But bootstrapping here>[*] lines of code on a real VAX is just never going to Steven> be very fast. There is no reason to blame GCC for that. Maybe. Then again, maybe there are real problems here. The ranlib one was already mentioned. And I wonder if libjava really needs to bring the host to its knees, as it does. I normally work on a 2 GHz Linux 2.6 system. It does lots of things quite fast. I can do large system builds and do other stuff at the same time while immense makefiles are crunching away. However, I can always tell when a GCC build has hit the libjava build -- that's when the *whole system* suddenly slows to a crawl. Maybe it comes from doing some processing on 5000 foo.o files all at once... :-( paul
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 27 Apr 2005, Steven Bosscher wrote: > [*] Does anyone have an idea of how large GCC really is? Using sloccount, 4.0.0 release looks like: Totals grouped by language (dominant language first): ansic: 1076327 (43.81%) ada: 541135 (22.03%) java:276544 (11.26%) cpp: 272101 (11.08%) sh: 222630 (9.06%) asm: 31194 (1.27%) yacc: 14900 (0.61%) exp: 8127 (0.33%) objc: 5919 (0.24%) fortran: 4507 (0.18%) perl: 1954 (0.08%) lex:768 (0.03%) awk:542 (0.02%) lisp:59 (0.00%) sed: 20 (0.00%) Total Physical Source Lines of Code (SLOC)= 2,456,727 Development Effort Estimate, Person-Years (Person-Months) = 725.95 (8,711.36) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) Schedule Estimate, Years (Months) = 6.55 (78.55) (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) Estimated Average Number of Developers (Effort/Schedule) = 110.90 Total Estimated Cost to Develop = $ 98,065,527 (average salary = $56,286/year, overhead = 2.40). Please credit this data as "generated using 'SLOCCount' by David A. Wheeler." Cheers, Karel -- Karel Gardas [EMAIL PROTECTED] ObjectSecurity Ltd. http://www.objectsecurity.com
Re: GCC 4.1: Buildable on GHz machines only?
> However, I can always tell when a GCC build has hit the libjava build > -- that's when the *whole system* suddenly slows to a crawl. Maybe > it comes from doing some processing on 5000 foo.o files all at > once... :-( But that is not GCC fault that binutils cannot handle that load. -- Pinski
Re: GCC 4.1: Buildable on GHz machines only?
On Apr 26, 2005, at 11:12 PM, Matt Thomas wrote: It would be nice if bootstrap emitted timestamps when it was started and when it completed a stage so one could just look at the make output. You can get them differenced for free by using: time make boostrap and written to a log file with nohup time make boostrap !
Re: GCC 4.1: Buildable on GHz machines only?
Mike Stump wrote: On Apr 26, 2005, at 11:12 PM, Matt Thomas wrote: It would be nice if bootstrap emitted timestamps when it was started and when it completed a stage so one could just look at the make output. You can get them differenced for free by using: time make boostrap I know that. But it's only works overall. I want the per-stage times. Here's a sparc64--netbsd full bootstrap including libjava (the machine has 640MB and was doing nothing but building gcc): 25406.01 real 21249.17 user 6283.15 sys 0 maximum resident set size 0 average shared memory size 0 average unshared data size 0 average unshared stack size 54689526 page reclaims 5349 page faults 110 swaps 723 block input operations 377302 block output operations 52 messages sent 52 messages received 285329 signals received 1037478 voluntary context switches 253151 involuntary context switches -- Matt Thomas email: [EMAIL PROTECTED] 3am Software Foundry www: http://3am-software.com/bio/matt/ Cupertino, CA disclaimer: I avow all knowledge of this message.
Re: GCC 4.1: Buildable on GHz machines only?
> "Andrew" == Andrew Pinski <[EMAIL PROTECTED]> writes: >> However, I can always tell when a GCC build has hit the libjava >> build -- that's when the *whole system* suddenly slows to a crawl. >> Maybe it comes from doing some processing on 5000 foo.o files all >> at once... :-( Andrew> But that is not GCC fault that binutils cannot handle that Andrew> load. Perhaps not. But perhaps there are workarounds that allow the gcc build to do its job without using binutils in a way that stresses it beyond its ability to cope. Matt's time output suggests that massive pagefaulting is a big issue -- and it wouldn't surprise me if the libjava build procedure were a major contributor there. paul
Re: GCC 4.1: Buildable on GHz machines only?
> > > "Andrew" == Andrew Pinski <[EMAIL PROTECTED]> writes: > > >> However, I can always tell when a GCC build has hit the libjava > >> build -- that's when the *whole system* suddenly slows to a crawl. > >> Maybe it comes from doing some processing on 5000 foo.o files all > >> at once... :-( > > Andrew> But that is not GCC fault that binutils cannot handle that > Andrew> load. > > Perhaps not. But perhaps there are workarounds that allow the gcc > build to do its job without using binutils in a way that stresses it > beyond its ability to cope. Well on ppc-darwin, linking libjava takes almost no time with only 256MB of memory. What times the time on ppc-darwin with libjava is libtool generating the list of files to link. So again this is a binutils problem as it works just fine with a good linker. -- Pinski
Re: GCC 4.1: Buildable on GHz machines only?
Steven Bosscher wrote: On Wednesday 27 April 2005 17:45, Matt Thomas wrote: The features under discussion are new, they didn't exist before. And because they never existed before, their cost for older platforms may not have been correctly assessed. If someone had cared about them, it would have been noticed earlier. But since _nobody_ has complained before you, I guess we can conclude that by far the majority if GCC users are quite happy with the cost assesments that were made. No, there have been plenty of complaints, but the GCC mailing lists have, shall we say, a "reputation", and a great many users will not post to them, either for fear of being ridiculed, or in the expection that they will not be heard. (Everything is archived, and they can see what happens to others.) If you want to see what users are *really* saying, look at the mailing lists of other projects, and see what is said when GCC comes up for discussion. Rightly or wrongly, the reward structure for GCC values new features on the latest hardware over speedy bootstrapping on old, so I don't expect things to change anytime soon. Stan
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote: > Steven Bosscher wrote: > > >On Wednesday 27 April 2005 17:45, Matt Thomas wrote: > > > >>>The features under discussion are new, they didn't exist before. > >>> > >>And because they never existed before, their cost for older platforms > >>may not have been correctly assessed. > >> > > > >If someone had cared about them, it would have been noticed earlier. > >But since _nobody_ has complained before you, I guess we can conclude > >that by far the majority if GCC users are quite happy with the cost > >assesments that were made. > > > No, there have been plenty of complaints, but the GCC mailing > lists have, shall we say, a "reputation", and a great many > users will not post to them, I've never in my life heard this from another mailing list, and i contribute to a *great* many open source projects. > either for fear of being ridiculed, > or in the expection that they will not be heard. (Everything is > archived, and they can see what happens to others.) The only person i see ridiculing people frequently happens to be from @apple.com. > If you want > to see what users are *really* saying, look at the mailing lists > of other projects, and see what is said when GCC comes up for > discussion. I do, and most appreciate the work we do. > > Rightly or wrongly, the reward structure for GCC values new > features on the latest hardware over speedy bootstrapping on > old, so I don't expect things to change anytime soon.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 03:13:21PM -0700, Stan Shebs wrote: > No, there have been plenty of complaints, but the GCC mailing > lists have, shall we say, a "reputation", and a great many > users will not post to them, either for fear of being ridiculed, > or in the expection that they will not be heard. (Everything is > archived, and they can see what happens to others.) Oh, come on, Stan. Compared to what? The GCC lists are quite civil compared to many free software developer lists. I'd say criticism of proposals is harsher on the Linux kernel list (though most of that criticism is fair). And if you *really* want to see a hostile environment, try debian-devel. That said, the gcc list is not intended to be a user support list, and users who come asking basic questions are (properly) asked to go elsewhere. We should make sure that we do this in a polite way, though. > Rightly or wrongly, the reward structure for GCC values new > features on the latest hardware over speedy bootstrapping on > old, so I don't expect things to change anytime soon. I will agree with you on this point, but more than half of the time to bootstrap consists of the time to build the Java library, and speeding that up is a losing battle, as Sun keeps adding new stuff that libgjc/classpath is then expected to clone, and the addition rate seems to exceed Moore's Law.
Re: GCC 4.1: Buildable on GHz machines only?
Daniel Berlin <[EMAIL PROTECTED]> writes: > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote: >> Steven Bosscher wrote: >> >If someone had cared about them, it would have been noticed >> >earlier. But since _nobody_ has complained before you, I guess we >> >can conclude that by far the majority if GCC users are quite happy >> >with the cost assesments that were made. >> > >> No, there have been plenty of complaints, but the GCC mailing lists >> have, shall we say, a "reputation", and a great many users will not >> post to them, > > I've never in my life heard this from another mailing list, and i > contribute to a *great* many open source projects. I have seen such complaints. Not about bootstrap times, no, that only affects people who compile the compiler; but the more general case of 'gcc takes forever to compile this program' does appear on a regular basis. I do also think that the amount of ridicule heaped on people who come to the gcc lists is, in general, too high. People should not be ridiculed for complaining that the compiler is slow, even if they are insisting on using vintage hardware. It is slow, even on fast hardware; it's just easier to see that on slow hardware. Rather more importantly, people should not be ridiculed for submitting bug reports, even if they are wrong. I suspect the bad public image that Stan refers to, has more to do with this than anything else. To be fair, it can be hard to explain that someone is wrong without belittling them; but it is important to make the effort, particularly when the reason they are wrong is esoteric (e.g. any of the legion of counterintuitive things in the C standard). Some people are worthy of ridicule; those who are trolling, or those who are persistently and wilfully clueless. However, ridicule is *not* an effective way of making these people shut up and go away, which is the appropriate goal when dealing with them. It can be fun to argue with them, but long argument threads are a drag on the mailing list, so we should not make a habit of it. This is not alt.flame. > The only person i see ridiculing people frequently happens to be > from @apple.com. I can think of four people who regularly ridicule posters. Only two of them are Apple employees. zw
Re: GCC 4.1: Buildable on GHz machines only?
Having seen Joe's comment, I should say that I agree with him that a lot of other projects' mailing lists are worse. However, that isn't an excuse in my book. zw
Re: GCC 4.1: Buildable on GHz machines only?
The alternative of course is to do only crossbuilds. Is it reasonable to say that, for platforms where a bootstrap is no longer feasible, a successful crossbuild is an acceptable test procedure to use instead? Sure, and get flamed and trounced by Uli on glibc when you talk about problems with crossbuilding. He'll flame you anyway for talking about platforms that are no longer relevant to the desktop. (Well, maybe he wouldn't. See http://lwn.net/Articles/19297 ) A successful crossbuild is certainly the minimum concievable standard. Perhaps one should also require bootstrapping the C compiler alone; that would provide at least some sanity-checking.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote: > Daniel Berlin <[EMAIL PROTECTED]> writes: > > > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote: > >> Steven Bosscher wrote: > >> >If someone had cared about them, it would have been noticed > >> >earlier. But since _nobody_ has complained before you, I guess we > >> >can conclude that by far the majority if GCC users are quite happy > >> >with the cost assesments that were made. > >> > > >> No, there have been plenty of complaints, but the GCC mailing lists > >> have, shall we say, a "reputation", and a great many users will not > >> post to them, > > > > I've never in my life heard this from another mailing list, and i > > contribute to a *great* many open source projects. > > I have seen such complaints. Not about bootstrap times, no, that only > affects people who compile the compiler; but the more general case of > 'gcc takes forever to compile this program' does appear on a regular > basis. People would complain even if the compiler took 1 second on every file, regardless of size or optimization level. If you want a faster compiler, it's hard work. It means not adding features because the design isn't a good one, *even if the user would still find it useful*. People aren't willing to do this. It means lots and lots of profiling, and taking care of little inefficiencies. All I ever see people suggest is magic bullets. We also have some deep datastructure problems in terms of IL, but those aren't going to give us a 5000% speedup or be a magic bullet either. --Dan
Re: GCC 4.1: Buildable on GHz machines only?
Daniel Berlin <[EMAIL PROTECTED]> writes: > On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote: >> I have seen such complaints. Not about bootstrap times, no, that only >> affects people who compile the compiler; but the more general case of >> 'gcc takes forever to compile this program' does appear on a regular >> basis. > > People would complain even if the compiler took 1 second on every file, > regardless of size or optimization level. Well, yes. 1 second/file is still slow! I want "make" to complete instantaneously! Don't you? > If you want a faster compiler, it's hard work. It means not adding > features because the design isn't a good one, *even if the user > would still find it useful*. People aren't willing to do this. It > means lots and lots of profiling, and taking care of little > inefficiencies. All I ever see people suggest is magic bullets. > > We also have some deep datastructure problems in terms of IL, but > those aren't going to give us a 5000% speedup or be a magic bullet > either. What you say is true. Does that mean we shouldn't try? zw
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 17:10 -0700, Zack Weinberg wrote: > Daniel Berlin <[EMAIL PROTECTED]> writes: > > On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote: > >> I have seen such complaints. Not about bootstrap times, no, that only > >> affects people who compile the compiler; but the more general case of > >> 'gcc takes forever to compile this program' does appear on a regular > >> basis. > > > > People would complain even if the compiler took 1 second on every file, > > regardless of size or optimization level. > > Well, yes. 1 second/file is still slow! I want "make" to complete > instantaneously! Don't you? > > > If you want a faster compiler, it's hard work. It means not adding > > features because the design isn't a good one, *even if the user > > would still find it useful*. People aren't willing to do this. It > > means lots and lots of profiling, and taking care of little > > inefficiencies. All I ever see people suggest is magic bullets. > > > > We also have some deep datastructure problems in terms of IL, but > > those aren't going to give us a 5000% speedup or be a magic bullet > > either. > > What you say is true. Does that mean we shouldn't try? Let me point out the important part again: All I ever see people suggest is magic bullets. We should try, but by doing the hard work. Not by expecting magic. > > zw
Re: GCC 4.1: Buildable on GHz machines only?
Daniel Berlin <[EMAIL PROTECTED]> writes: >> What you say is true. Does that mean we shouldn't try? > > Let me point out the important part again: All I ever see people > suggest is magic bullets. > > We should try, but by doing the hard work. Not by expecting magic. Sure. CodeSourcery did have a contract last year to do some of that hard work, and there might be another one in the pipe (tho it probably won't come through until after I leave). Each individual piece isn't that hard, either, although they also tend not to produce easurable gains (but if you do lots of them, they add up...) Sadly, I have no time these days for any GCC work other than what I'm being paid to do. zw
Re: GCC 4.1: Buildable on GHz machines only?
>>> The alternative of course is to do only crossbuilds. Is it reasonable >>> to say that, for platforms where a bootstrap is no longer feasible, a >>> successful crossbuild is an acceptable test procedure to use instead? >>> >> Sure, and get flamed and trounced by Uli on glibc when you talk >> about problems with crossbuilding. > >He'll flame you anyway for talking about platforms >that are no longer relevant to the desktop. >(Well, maybe he wouldn't. See http://lwn.net/Articles/19297 ) > >A successful crossbuild is certainly the minimum concievable standard. >Perhaps one should also require bootstrapping the C compiler alone; >that would provide at least some sanity-checking. Unfortunately for some of the embedded targets(like the ColdFire V4e work I'm doing), a bootstrap is impossible due to limited memory and no usable mass-storage device on the hardware I have available, so hopefully a successful crossbuild will suffice. -- Peter Barada [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
>Well, yes. 1 second/file is still slow! I want "make" to complete >instantaneously! Don't you? Actually I want it to complete before I even start, but I don't want to get too greedy. :) What's really sad is that for cross-compilation of the toolchain, we have to repeat a few steps (build gcc twice, build glibc twice) because glibc and gcc assume that a near-complete environment is available(such as gcc needing headers, and glibc needing -lgcc-eh), so even really fast machines(2.4Ghz P4) take an hour to do a cross-build from scratch. -- Peter Barada [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
On 2005-04-27, at 21:57, Steven Bosscher wrote: On Wednesday 27 April 2005 17:45, Matt Thomas wrote: The features under discussion are new, they didn't exist before. And because they never existed before, their cost for older platforms may not have been correctly assessed. If someone had cared about them, it would have been noticed earlier. But since _nobody_ has complained before you, I guess we can conclude that by far the majority if GCC users are quite happy with the cost assesments that were made. That is simply not true. I don't feel happy about the whole gcj thing. Already libstdc++ provided alongside with the compiler is pain enough. But the runtime support for Java is a disaster in the making since it's getting to be a truly huge behemoth which is constantly changing and "eating" new APIs. Worst thing about it is it's grief for external libraries like GTK+ for example.
Re: GCC 4.1: Buildable on GHz machines only?
> "Paul" == Paul Koning <[EMAIL PROTECTED]> writes: Paul> Maybe. Then again, maybe there are real problems here. The ranlib Paul> one was already mentioned. And I wonder if libjava really needs to Paul> bring the host to its knees, as it does. Killing machines is only a secondary goal, if that's what you mean ;-) The bad news is that libjava is only going to grow. On the other hand, while I haven't measured it myself, I've heard that a lot of the time in the libjava build is spent in libtool (versus plain old ld). Perhaps that can be alleviated somehow. It is getting to be a major problem, to the point where it is simpler to do one's development in Classpath and avoid libgcj just because of the terrible turnaround time. Unfortunately it seems to be an easier problem to diagnose than to cure. Tom
Re: GCC 4.1: Buildable on GHz machines only?
On 2005-04-28, at 03:06, Peter Barada wrote: Well, yes. 1 second/file is still slow! I want "make" to complete instantaneously! Don't you? Actually I want it to complete before I even start, but I don't want to get too greedy. :) What's really sad is that for cross-compilation of the toolchain, we have to repeat a few steps (build gcc twice, build glibc twice) because glibc and gcc assume that a near-complete environment is available(such as gcc needing headers, and glibc needing -lgcc-eh), so even really fast machines(2.4Ghz P4) take an hour to do a cross-build from scratch. Actually what GCC needs to know are the following: 1. What will the signal strucutre look alike on the target system. In esp.: What does the kernel think? What does the glibc think? 2. Do we do TLS? 3. Do we do linuxthreads or nptl? glibc just wants: 1. Say hello to libgcc_s 2. Does the compiler support TLS? And then don't forget that libgcc_s wants: 1. Say hello to C++, in a way requiring libc functions for exception handling. With a "tad bit" of work the double compilation can be avoided for the glibc. You will have to build a GCC with static libgcc first and you will only need the second gcc build cycle to get a dynamic libgcc_s as well as C++, since C++ makes a shared libgcc mandatory. The whole double build could be avoided if: 1. It would be possible to build libgcc for itself without rebuilding the whole compiler. 2. It would be possible to build first the C compiler and then just the C++ compiler. 2. The information required by glibc could be provided statically to it. All of the above are basically problems of the "configure" system - which isn't pretty.
Re: GCC 4.1: Buildable on GHz machines only?
On 2005-04-28, at 01:35, Joe Buck wrote: I will agree with you on this point, but more than half of the time to bootstrap consists of the time to build the Java library, and speeding that up is a losing battle, as Sun keeps adding new stuff that libgjc/classpath is then expected to clone, and the addition rate seems to exceed Moore's Law. Those are just symptoms of the fact that Java is an emulated machine with his own OS, which just happened to forget about any lesson in this area learned during the last 30 years. Other then this: please.please.bePolite.toThe.niceFine.javaApi(nowAndAlways);
Re: GCC 4.1: Buildable on GHz machines only?
On 2005-04-27, at 22:54, Karel Gardas wrote: Total Physical Source Lines of Code (SLOC)= 2,456,727 Development Effort Estimate, Person-Years (Person-Months) = 725.95 (8,711.36) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) Schedule Estimate, Years (Months) = 6.55 (78.55) (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) Estimated Average Number of Developers (Effort/Schedule) = 110.90 Total Estimated Cost to Develop = $ 98,065,527 (average salary = $56,286/year, overhead = 2.40). Please credit this data as "generated using 'SLOCCount' by David A. Wheeler." One question remains open: Who is Mr. Person?
Re: GCC 4.1: Buildable on GHz machines only?
Peter Barada <[EMAIL PROTECTED]> writes: > What's really sad is that for cross-compilation of the toolchain, we > have to repeat a few steps (build gcc twice, build glibc twice) > because glibc and gcc assume that a near-complete environment is > available(such as gcc needing headers, and glibc needing -lgcc-eh), so > even really fast machines(2.4Ghz P4) take an hour to do a cross-build > from scratch. This could be made substantially easier if libgcc moved to the top level. You wanna help out with that? zw
Re: GCC 4.1: Buildable on GHz machines only?
>> What's really sad is that for cross-compilation of the toolchain, we >> have to repeat a few steps (build gcc twice, build glibc twice) >> because glibc and gcc assume that a near-complete environment is >> available(such as gcc needing headers, and glibc needing -lgcc-eh), so >> even really fast machines(2.4Ghz P4) take an hour to do a cross-build >> from scratch. > >This could be made substantially easier if libgcc moved to the top >level. You wanna help out with that? Uh, ok. What do you mean by "move to the top level"? -- Peter Barada [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
Peter Barada <[EMAIL PROTECTED]> writes: >>> What's really sad is that for cross-compilation of the toolchain, we >>> have to repeat a few steps (build gcc twice, build glibc twice) >>> because glibc and gcc assume that a near-complete environment is >>> available(such as gcc needing headers, and glibc needing -lgcc-eh), so >>> even really fast machines(2.4Ghz P4) take an hour to do a cross-build >>> from scratch. >> >>This could be made substantially easier if libgcc moved to the top >>level. You wanna help out with that? > > Uh, ok. What do you mean by "move to the top level"? libgcc should have its own top-level directory like all the rest of the runtime libraries, instead of being nested inside the gcc directory. This helps because it's libgcc, not the compiler itself, that needs the libc headers. You could build your cross-gcc, use it to configure libc to the point where the headers are available, build libgcc, then go back and finish building libc, and not have to repeat anything. At least that's the theory. I'm sure there are all kinds of practical problems that will crop up when we get to the point we can actually try this, but it's a long step in the right direction. I'm not sure who is working on this now, but Paolo Bonzini should know. zw
Re: GCC 4.1: Buildable on GHz machines only?
Zack Weinberg dixit: >This could be made substantially easier if libgcc moved to the top >level. You wanna help out with that? What about crtstuff? //mirabile
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 27 Apr 2005, Daniel Berlin wrote: > If you want a faster compiler, it's hard work. It means not adding > features because the design isn't a good one, *even if the user would > still find it useful*. People aren't willing to do this. It means lots > and lots of profiling, and taking care of little inefficiencies. All I > ever see people suggest is magic bullets. Daniel, I can not agree with it at all! At least for our C++ code, I've seen compiler speedup from GCC 3.2, which was the most slowest compiler, 3.3 was faster because of better memory heuristics, 3.4 was faster probably because of better parser and 4.0 is faster because of a work of many developers here whom I would like to say Thank You! Imagine that G++ is now faster at least about 50% (at least!), when comparing 3.2 and 4.0.0, not just talking about great 25% speedup between 3.4.x and 4.0.0! Conclusion: people are willing to investigate compiler slowness and even they add new features to the compiler itself. Thank you all for writing GCC! Karel -- Karel Gardas [EMAIL PROTECTED] ObjectSecurity Ltd. http://www.objectsecurity.com
Re: GCC 4.1: Buildable on GHz machines only?
Paul Koning writes: > > "Andrew" == Andrew Pinski <[EMAIL PROTECTED]> writes: > > >> However, I can always tell when a GCC build has hit the libjava > >> build -- that's when the *whole system* suddenly slows to a crawl. > >> Maybe it comes from doing some processing on 5000 foo.o files all > >> at once... :-( > > Andrew> But that is not GCC fault that binutils cannot handle that > Andrew> load. > > Perhaps not. But perhaps there are workarounds that allow the gcc > build to do its job without using binutils in a way that stresses it > beyond its ability to cope. > > Matt's time output suggests that massive pagefaulting is a big issue I'm sure that's true. The working set exceeds the amount of RAM in the system, leading to near worst-case behaviour. > -- and it wouldn't surprise me if the libjava build procedure were a > major contributor there. Yes. This is a profile of the libgcj build. The single biggest user of CPU is the libtool shell script, which uses more than a quarter of the total (non-kernel) CPU time. However, it's important not to be misled -- I'm sure the linker causes a huge amount of disk activity, so it's not just CPU time that is important. Having said that, I suspect that the single biggest improvement to the libgcj build time would be either to remove the libtool shell script altogether or find some way to reduce its use or make it faster. CPU: CPU with timer interrupt, speed 0 MHz (estimated) Profiling through timer interrupt TIMER:0| samples| %| -- 1770547 63.0596 no-vmlinux 415708 14.8058 libc-2.3.4.so 259889 9.2562 ltsh 257355 9.1659 jc1 22111 0.7875 cc1plus 20260 0.7216 as 19289 0.6870 ld-2.3.4.so 10502 0.3740 make 5921 0.2109 sed 5163 0.1839 libbfd-2.15.92.0.2.so 2855 0.1017 gcj 2724 0.0970 cc1 2218 0.0790 libz.so.1.2.1.2 2154 0.0767 grep 2019 0.0719 xterm 1864 0.0664 ld Andrew.
RE: GCC 4.1: Buildable on GHz machines only?
Original Message >From: Marcin Dalecki >Sent: 28 April 2005 02:58 > On 2005-04-27, at 22:54, Karel Gardas wrote: >> >> Total Physical Source Lines of Code (SLOC)= 2,456,727 >> Development Effort Estimate, Person-Years (Person-Months) = 725.95 >> (8,711.36) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) >> Schedule Estimate, Years (Months) = 6.55 (78.55) >> (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) >> Estimated Average Number of Developers (Effort/Schedule) = 110.90 >> Total Estimated Cost to Develop = $ 98,065,527 >> (average salary = $56,286/year, overhead = 2.40). >> Please credit this data as "generated using 'SLOCCount' by David A. >> Wheeler." > > One question remains open: Who is Mr. Person? What makes you think it's not Ms. Person? Chauvinist! ;) cheers, DaveK -- Can't think of a witty .sigline today
Re: GCC 4.1: Buildable on GHz machines only?
On 2005-04-28, at 12:03, Dave Korn wrote: Original Message From: Marcin Dalecki Sent: 28 April 2005 02:58 On 2005-04-27, at 22:54, Karel Gardas wrote: Total Physical Source Lines of Code (SLOC)= 2,456,727 Development Effort Estimate, Person-Years (Person-Months) = 725.95 (8,711.36) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) Schedule Estimate, Years (Months) = 6.55 (78.55) (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) Estimated Average Number of Developers (Effort/Schedule) = 110.90 Total Estimated Cost to Develop = $ 98,065,527 (average salary = $56,286/year, overhead = 2.40). Please credit this data as "generated using 'SLOCCount' by David A. Wheeler." One question remains open: Who is Mr. Person? What makes you think it's not Ms. Person? Chauvinist! Oh indeed... Yes I missed it: It's Lady Wheeler!
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 20:57, Steven Bosscher wrote: > On Wednesday 27 April 2005 17:45, Matt Thomas wrote: > > > The features under discussion are new, they didn't exist before. > > > > And because they never existed before, their cost for older platforms > > may not have been correctly assessed. > > If someone had cared about them, it would have been noticed earlier. > But since _nobody_ has complained before you, I guess we can conclude > that by far the majority if GCC users are quite happy with the cost > assesments that were made. This is false. I've been complaining (at various levels of volume) ever since we switched to using the garbage collector.[1] R. [1] I don't think the Garbage Collector is the only source of slowdown. It was just the change that tipped the balance from not-very good to insane. Unfortunately, things have continued to go down-hill from there.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 21:55, Andrew Pinski wrote: > > However, I can always tell when a GCC build has hit the libjava build > > -- that's when the *whole system* suddenly slows to a crawl. Maybe > > it comes from doing some processing on 5000 foo.o files all at > > once... :-( > > But that is not GCC fault that binutils cannot handle that load. > > -- Pinski It's not as simple as just saying "binutils should be able to cope with any number of objects thrown at it". Part of the problem is that 5000 object files exceeds the system limits of the host machine (eg command line length, etc). R.
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, Apr 28, 2005 at 07:31:06AM +, Thorsten Glaser wrote: > Zack Weinberg dixit: > > >This could be made substantially easier if libgcc moved to the top > >level. You wanna help out with that? > > What about crtstuff? Yes, they should be moved at the same time; I consider them closer to part of libgcc than to part of gcc proper. -- Daniel Jacobowitz CodeSourcery, LLC
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 2005-04-28 at 02:40, Tom Tromey wrote: > > "Paul" == Paul Koning <[EMAIL PROTECTED]> writes: > > Paul> Maybe. Then again, maybe there are real problems here. The ranlib > Paul> one was already mentioned. And I wonder if libjava really needs to > Paul> bring the host to its knees, as it does. > > Killing machines is only a secondary goal, if that's what you mean ;-) > > The bad news is that libjava is only going to grow. > > On the other hand, while I haven't measured it myself, I've heard that > a lot of the time in the libjava build is spent in libtool (versus > plain old ld). Perhaps that can be alleviated somehow. > Splitting libjava into multiple libraries would be a good start. It would probably be a good thing for application start-up time to when using shared libs. R.
Re: GCC 4.1: Buildable on GHz machines only?
Richard Earnshaw writes: > On Wed, 2005-04-27 at 21:55, Andrew Pinski wrote: > > > However, I can always tell when a GCC build has hit the libjava build > > > -- that's when the *whole system* suddenly slows to a crawl. Maybe > > > it comes from doing some processing on 5000 foo.o files all at > > > once... :-( > > > > But that is not GCC fault that binutils cannot handle that load. > > > > -- Pinski > > It's not as simple as just saying "binutils should be able to cope with > any number of objects thrown at it". Part of the problem is that 5000 > object files exceeds the system limits of the host machine (eg command > line length, etc). If ld can't accept a list of files from a stream but is instead limited by command line length, then that *is* the fault of ld. Andrew.
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 2005-04-28 at 14:35, Andrew Haley wrote: > Richard Earnshaw writes: > > On Wed, 2005-04-27 at 21:55, Andrew Pinski wrote: > > > > However, I can always tell when a GCC build has hit the libjava build > > > > -- that's when the *whole system* suddenly slows to a crawl. Maybe > > > > it comes from doing some processing on 5000 foo.o files all at > > > > once... :-( > > > > > > But that is not GCC fault that binutils cannot handle that load. > > > > > > -- Pinski > > > > It's not as simple as just saying "binutils should be able to cope with > > any number of objects thrown at it". Part of the problem is that 5000 > > object files exceeds the system limits of the host machine (eg command > > line length, etc). > > If ld can't accept a list of files from a stream but is instead > limited by command line length, then that *is* the fault of ld. A certain proverb relating to bad workmen and tools springs to mind here. I think we really need to consider whether requiring ld to support linking of 5000 object files might mean poor library design. R.
Re: GCC 4.1: Buildable on GHz machines only?
Andrew Haley <[EMAIL PROTECTED]> writes: > If ld can't accept a list of files from a stream but is instead > limited by command line length, then that *is* the fault of ld. You can always use a linker script. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: GCC 4.1: Buildable on GHz machines only?
I have to agree with Richard's assessment, gcc is currently on the verge of being unusable in many instances. If you have a lot of software to build and have to do complete rebuilds it's painful, the binutils guys have a 3x speedup patch coming up, but every time there is a speedup it gets eaten up. gcc seem's to be moving more and more data around and does more and more temporary allocations of huge chunks of memory. Sometimes a concearn pop's up and some 5 to 10 percent is gained only to be lost again. My opinion on the point is that if there was a 100% speedup of gcc it would be very good but not enough since it has slowed down so much mainly from memory usage. I used to do kernel compiles in an hour or two on a pentium laptop, and now I can forget it since it's memory is soon exhausted and it goes into swap. I have never done any 'memory profiling' but I think it might be time to give it a shot, does anybody have any hints on how to go about something like this ? It should be enough to have a look at the places where were searching for information that we already have had, or to make any kind of search more efficient if possible. Perhaps it's possible to graft some kind of indexing scheme on the worst offenders, so that some O(n^2) can get to O(1 ) or something. The main problem seem's to be that almost any efforts to point out a single scapegoat have not been very successful. / regards, Lars Segerlund. On Thu, 28 Apr 2005 14:24:11 +0100 Richard Earnshaw <[EMAIL PROTECTED]> wrote: > On Wed, 2005-04-27 at 20:57, Steven Bosscher wrote: > > On Wednesday 27 April 2005 17:45, Matt Thomas wrote: > > > > The features under discussion are new, they didn't exist before. > > > > > > And because they never existed before, their cost for older platforms > > > may not have been correctly assessed. > > > > If someone had cared about them, it would have been noticed earlier. > > But since _nobody_ has complained before you, I guess we can conclude > > that by far the majority if GCC users are quite happy with the cost > > assesments that were made. > > This is false. I've been complaining (at various levels of volume) ever > since we switched to using the garbage collector.[1] > > R. > > [1] I don't think the Garbage Collector is the only source of slowdown. > It was just the change that tipped the balance from not-very good to > insane. Unfortunately, things have continued to go down-hill from > there.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 08:05:39AM -0700, Matt Thomas wrote: > Am I the only person who has attempted to do a native bootstrap on a > system as slow as a M68k? I am using an Amiga with [EMAIL PROTECTED] myself. My last GCC bootstrap on that machine was done in 1999 for GCC 2.95.2 and it took several hours. I never tried to boostrap newer GCC version because I fear that it would take much too long. I only did a non-optimized non-bootstrap build of 3.2.2 with 2.95.2 as build compiler on that machine. I have to cross-build GCC3 and newer to use them on this machine. Anything else wouldn't work. FWIW, I experienced a constant slowdown of the C compiler with every release since 2.7. While C is still usable, C++ with templates is not. Gunther
Re: GCC 4.1: Buildable on GHz machines only?
Peter Barada wrote: Well, yes. 1 second/file is still slow! I want "make" to complete instantaneously! Don't you? Actually I want it to complete before I even start, but I don't want to get too greedy. :) What's really sad is that for cross-compilation of the toolchain, we have to repeat a few steps (build gcc twice, build glibc twice) because glibc and gcc assume that a near-complete environment is available(such as gcc needing headers, and glibc needing -lgcc-eh), so even really fast machines(2.4Ghz P4) take an hour to do a cross-build from scratch. That sounds comparable to the time required to build RTEMS toolsets. I just looked at the timestamp on the build logs for a gcc 4.0.0 CVS build with newlib 1.13.0 and it is on the order of 60-90 minutes per target on a 2.4 Ghz P4 w/512 MB RAM. This is just C and C++ and the variance is probably mostly due to the number of multilibs. I checked build logs of gcc 3.3.5 with newlib 1.13 from December on the same machine and times were comparable so it isn't a recent slowdown. When I built my native 4.0.0 on this machine, I did notice that compiling the Java libraries took long enough to notice and when I did a top, I noticed that gjc was running for 30+ seconds of CPU time with RSS on the order of 150 MB. Another time I noticed that sh had taken 90 seconds and that was an invocation of libtool with a very long command line. Here is the configure command and output of time on this machine for a gcc 4.0.0 build: ../gcc-4.0.0/configure --prefix=/opt/gcc-4.0.0 --enable-languages=c,c++,ada,java,objc 4271.94user 685.49system 1:31:18elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k0inputs+0outputs (33176183major+40970199minor)pagefaults 0swaps A 2.4 Ghz P4 isn't what I would consider an obsolete machine and it took 90 minutes for "make" -- not a full bootstrap. --joel
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 04:40:29PM -0700, Zack Weinberg wrote: > Daniel Berlin <[EMAIL PROTECTED]> writes: > > > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote: > >> Steven Bosscher wrote: > >> >If someone had cared about them, it would have been noticed > >> >earlier. But since _nobody_ has complained before you, I guess we > >> >can conclude that by far the majority if GCC users are quite happy > >> >with the cost assesments that were made. > >> > > >> No, there have been plenty of complaints, but the GCC mailing lists > >> have, shall we say, a "reputation", and a great many users will not > >> post to them, > > > > I've never in my life heard this from another mailing list, and i > > contribute to a *great* many open source projects. > > I have seen such complaints. Not about bootstrap times, no, that only > affects people who compile the compiler; but the more general case of > 'gcc takes forever to compile this program' does appear on a regular > basis. Maybe there are less/no complaints about bootstrap times, because people that are able to make a bootstrap know that complaining doesn't help. My primary target is m68k and I never attempted a bootstrap of GCC3 there because it would take much to much time. Now I got used to cross-build GCC (only C and C++) for m68k-amigaos. And since this target isn't in the official tree its even more painful to inquire the list. > I do also think that the amount of ridicule heaped on people who come > to the gcc lists is, in general, too high. People should not be > ridiculed for complaining that the compiler is slow, even if they are > insisting on using vintage hardware. IMHO GCC3 is better than GCC2 and thus its worthwile to use it even on "vintage" hardware. > It is slow, even on fast hardware; it's just easier to see that on > slow hardware. I mainly use C and thats acceptable with GCC3 but eg. C++ with templates is really slow on my m68k Amiga. > Rather more importantly, people should not be ridiculed for submitting > bug reports, even if they are wrong. I suspect the bad public image > that Stan refers to, has more to do with this than anything else. FWIW, I fully agree. Gunther
Re: GCC 4.1: Buildable on GHz machines only?
>> What's really sad is that for cross-compilation of the toolchain, we >> have to repeat a few steps (build gcc twice, build glibc twice) >> because glibc and gcc assume that a near-complete environment is >> available(such as gcc needing headers, and glibc needing -lgcc-eh), so >> even really fast machines(2.4Ghz P4) take an hour to do a cross-build >> from scratch. > >That sounds comparable to the time required to build RTEMS toolsets. I >just looked at the timestamp on the build logs for a gcc 4.0.0 CVS build >with newlib 1.13.0 and it is on the order of 60-90 minutes per target on >a 2.4 Ghz P4 w/512 MB RAM. This is just C and C++ and the variance is >probably mostly due to the number of multilibs. This is for a m68k-linux build (with coldfire-linux config for glibc), and its only the C compiler, so adding C++ will obvioulsy make it take longer. >A 2.4 Ghz P4 isn't what I would consider an obsolete machine and it took >90 minutes for "make" -- not a full bootstrap. Even on a 3.0Ghz P4 with HT, 1Gb DDR and a hardware RAID with SATA drives it takes about 30 minutes so there's a *lot* of work going on, and I'd call that near cutting-edge. -- Peter Barada [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
Andreas Schwab writes: > Andrew Haley <[EMAIL PROTECTED]> writes: > > > If ld can't accept a list of files from a stream but is instead > > limited by command line length, then that *is* the fault of ld. > > You can always use a linker script. Yeah, good point. libtool seems to go to extraordinary lengths to avoid doing so, I presume because it isn't portable. Andrew.
Re: GCC 4.1: Buildable on GHz machines only?
> Andrew Haley writes: Andrew> Yeah, good point. libtool seems to go to extraordinary lengths to Andrew> avoid doing so, I presume because it isn't portable. Current libtool does allow a list of files, but the version used by GCC is not recent. David
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 2005-04-28 at 11:58 -0400, Peter Barada wrote: > This is for a m68k-linux build (with coldfire-linux config for glibc), > and its only the C compiler, so adding C++ will obvioulsy make it take > longer. > > >A 2.4 Ghz P4 isn't what I would consider an obsolete machine and it took > >90 minutes for "make" -- not a full bootstrap. > > Even on a 3.0Ghz P4 with HT, 1Gb DDR and a hardware RAID with SATA > drives it takes about 30 minutes so there's a *lot* of work going on, > and I'd call that near cutting-edge. > 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of yesterday. 2. Building XLC with (C,C++,Fortran) and a single backend takes roughly the same time as building GCC. And they aren't three staging, AFAIK. --Dan
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote: > Daniel Berlin <[EMAIL PROTECTED]> writes: > > > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote: > >> Steven Bosscher wrote: > >> >If someone had cared about them, it would have been noticed > >> >earlier. But since _nobody_ has complained before you, I guess we > >> >can conclude that by far the majority if GCC users are quite happy > >> >with the cost assesments that were made. > >> > > >> No, there have been plenty of complaints, but the GCC mailing lists > >> have, shall we say, a "reputation", and a great many users will not > >> post to them, > > > > I've never in my life heard this from another mailing list, and i > > contribute to a *great* many open source projects. > > I have seen such complaints. Not about bootstrap times, no, that only > affects people who compile the compiler; but the more general case of > 'gcc takes forever to compile this program' does appear on a regular > basis. > > I do also think that the amount of ridicule heaped on people who come > to the gcc lists is, in general, too high. People should not be > ridiculed for complaining that the compiler is slow, even if they are > insisting on using vintage hardware. It is slow, even on fast > hardware; it's just easier to see that on slow hardware. For people who use hardware most developers don't have access to, we need preprocessed source and profiling data, or else there is no chance of fixing it. It seems users of these platforms are not willing to provide this, even when asked. Those people who have provided preprocessed source and profiling data (8361, omniorb, etc) have had their compilation times sped up. So whenever i hear that "we don't care about compilation time", i wonder if the user has even put code in bugzilla that demonstrates the problem. Most often, the answer is no, AFAICT. --Dan
Re: GCC 4.1: Buildable on GHz machines only?
Andrew Haley <[EMAIL PROTECTED]> writes: > If ld can't accept a list of files from a stream but is instead > limited by command line length, then that *is* the fault of ld. GNU ld won't currently read a list of files from stdin, but it will read a list of files from a file. For example, look at /usr/lib/libc.so on a typical GNU/Linux system. Ian
Re: GCC 4.1: Buildable on GHz machines only?
Ian Lance Taylor writes: > Andrew Haley <[EMAIL PROTECTED]> writes: > > > If ld can't accept a list of files from a stream but is instead > > limited by command line length, then that *is* the fault of ld. > > GNU ld won't currently read a list of files from stdin, but it will > read a list of files from a file. > > For example, look at /usr/lib/libc.so on a typical GNU/Linux system. Yes thanks, I've had that pointed out to me. Apparently the real issue here is that we have an older version of libtool in the gcc tree. Andrew.
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, Apr 27, 2005 at 07:40:37PM -0600, Tom Tromey wrote: > > "Paul" == Paul Koning <[EMAIL PROTECTED]> writes: > > Paul> Maybe. Then again, maybe there are real problems here. The ranlib > Paul> one was already mentioned. And I wonder if libjava really needs to > Paul> bring the host to its knees, as it does. > > Killing machines is only a secondary goal, if that's what you mean ;-) > > The bad news is that libjava is only going to grow. > > On the other hand, while I haven't measured it myself, I've heard that > a lot of the time in the libjava build is spent in libtool (versus > plain old ld). Perhaps that can be alleviated somehow. Has anyone looked at oprofile data for the libjava build?
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, Apr 28, 2005 at 12:09:35PM -0400, David Edelsohn wrote: > > Andrew Haley writes: > > Andrew> Yeah, good point. libtool seems to go to extraordinary lengths to > Andrew> avoid doing so, I presume because it isn't portable. > > Current libtool does allow a list of files, but the version used > by GCC is not recent. Is there a reason why we aren't using a recent libtool?
Re: GCC 4.1: Buildable on GHz machines only?
> Andrew Haley writes: Andrew> Yes thanks, I've had that pointed out to me. Apparently the real Andrew> issue here is that we have an older version of libtool in the gcc Andrew> tree. Any feature in libtool CVS is fair game to be backported to libtool in GCC. I am planning to backport a subset of the feature for AIX. I can backport it all, including the GNU ld support, if you want. David
Re: GCC 4.1: Buildable on GHz machines only?
> Joe Buck writes: Joe> Is there a reason why we aren't using a recent libtool? Porting and testing effort to upgrade. David
Re: GCC 4.1: Buildable on GHz machines only?
Joe Buck writes: > On Wed, Apr 27, 2005 at 07:40:37PM -0600, Tom Tromey wrote: > > > "Paul" == Paul Koning <[EMAIL PROTECTED]> writes: > > > > Paul> Maybe. Then again, maybe there are real problems here. The ranlib > > Paul> one was already mentioned. And I wonder if libjava really needs to > > Paul> bring the host to its knees, as it does. > > > > Killing machines is only a secondary goal, if that's what you mean ;-) > > > > The bad news is that libjava is only going to grow. > > > > On the other hand, while I haven't measured it myself, I've heard that > > a lot of the time in the libjava build is spent in libtool (versus > > plain old ld). Perhaps that can be alleviated somehow. > > Has anyone looked at oprofile data for the libjava build? Again: CPU: CPU with timer interrupt, speed 0 MHz (estimated) Profiling through timer interrupt TIMER:0| samples| %| -- 1770547 63.0596 no-vmlinux 415708 14.8058 libc-2.3.4.so 259889 9.2562 ltsh 257355 9.1659 jc1 22111 0.7875 cc1plus 20260 0.7216 as 19289 0.6870 ld-2.3.4.so 10502 0.3740 make 5921 0.2109 sed 5163 0.1839 libbfd-2.15.92.0.2.so 2855 0.1017 gcj 2724 0.0970 cc1 2218 0.0790 libz.so.1.2.1.2 2154 0.0767 grep 2019 0.0719 xterm 1864 0.0664 ld Andrew.
Re: GCC 4.1: Buildable on GHz machines only?
On 2005-04-28, at 16:26, Lars Segerlund wrote: I have never done any 'memory profiling' but I think it might be time to give it a shot, does anybody have any hints on how to go about something like this ? The main performance problem for GCC as I see it is structural. GCC is emulating the concept of high polymorphism of data structures in plain C. Well actually it's not even plain C any longer... GTY(). Using C++ with simple inheritance could help *a lot* here. (Duck...) There is too much of pointer indirection in the data structures too. Look throughout the code and wonder how very few iterations go around plan, simple, fast, spare on register set, allowing cache prefetch, to work C arrays. GCC is using linked lists to test the "efficiency" of TLB and other mechanisms of the CPU.
Re: GCC 4.1: Buildable on GHz machines only?
On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote: 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of yesterday. 2. Building XLC with (C,C++,Fortran) and a single backend takes roughly the same time as building GCC. And they aren't three staging, AFAIK. "..ain't the same ballpark, it ain't the same league, it ain't even the same..." :-) - Devang
Re: GCC 4.1: Buildable on GHz machines only?
On Wed, 27 Apr 2005 16:52:25 -0400, Paul Koning <[EMAIL PROTECTED]> said: > However, I can always tell when a GCC build has hit the libjava build > -- that's when the *whole system* suddenly slows to a crawl. Maybe > it comes from doing some processing on 5000 foo.o files all at > once... :-( It's also too bad the final steps of the libjava build aren't more parallelizable, though I can't say I have any productive suggestions to add there. I just tried a C/C++ bootstrap and a C/C++/Java bootstrap on a four-processor machine; the latter took 2.6 times as long as the former, and for most of the non-compilation part of the libjava build, three of the processors were sitting around being bored. (Not that I really know exactly what was causing the delays; maybe the disk and memory were being stressed enough by the single processor that was active.) This also showed up a little in the C build: while make found other stuff to do while gen-attrtab was going on, shortly after it started compiling insn-attrtab.c, make ran out of other files to compile. Not that I'm really complaining: you can get quite a lot of mileage out of multiple CPUs as it is, more than enough (in my opinion) to justify purchasing some nice build servers by software shops that do a lot of GCC work. (I won't post the actual bootstrap times out of fear of being lynched.) This might show up more as people start moving towards dual-core and/or multiple CPU systems even on the low end. David Carlton [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 2005-04-28 at 10:23 -0700, Devang Patel wrote: > On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote: > > > 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of > > yesterday. > > 2. Building XLC with (C,C++,Fortran) and a single backend takes > > roughly > > the same time as building GCC. And they aren't three staging, AFAIK. > > "..ain't the same ballpark, it ain't the same league, > it ain't even the same..." :-) Bullshit. They are both production quality compilers. People complain gcc bootstrap is too long. XLC at O2 (which is what they compile at, last i looked) isn't running the IPA middle end, etc. --Dan
Re: GCC 4.1: Buildable on GHz machines only?
>Not that I'm really complaining: you can get quite a lot of mileage >out of multiple CPUs as it is, more than enough (in my opinion) to >justify purchasing some nice build servers by software shops that do a >lot of GCC work. (I won't post the actual bootstrap times out of fear >of being lynched.) This might show up more as people start moving >towards dual-core and/or multiple CPU systems even on the low end. That's great if the software can be cross-built. As it is, cross-building a toolchain requires a lot of extra work, and if it weren't for Dan Kegel's commitment, I'd dare say near impossible. I've watched the sometimes near-indifference to the problems we have trying to put together toolchains for non-hosted environments. Even when I have a cross-toolchain, its still a *long* uphill battle since there are too many OSS packages out there that can't cross-configure/compile (openssh, perl as examples off the top of my head) without a *lot* of work. Its just that it takes a lot of time and work to cross-build a non-x86 linux environment to verify any changes in the toolchain. And comments like "get a faster machine" are a non-starter. -- Peter Barada [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
Someone complained I was unfair in my gcc bootstrap times since some builds included libjava/gfortran and some did not. So in the past day, I've done bootstrap with just c,c++,objc on both 3.4 and gcc4.1. I've put the results in a web page at http://3am-software.com/gcc-speed.html. The initial bootstrap compiler was gcc3.3 and they are all running off the same base of NetBSD 3.99.3. While taking out fortran and java reduced the disparity, there is still a large increase in bootstrap times from 3.4 to 4.1. -- Matt Thomas email: [EMAIL PROTECTED] 3am Software Foundry www: http://3am-software.com/bio/matt/ Cupertino, CA disclaimer: I avow all knowledge of this message.
Re: GCC 4.1: Buildable on GHz machines only?
On Apr 28, 2005, at 10:54 AM, Daniel Berlin wrote: On Thu, 2005-04-28 at 10:23 -0700, Devang Patel wrote: On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote: 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of yesterday. 2. Building XLC with (C,C++,Fortran) and a single backend takes roughly the same time as building GCC. And they aren't three staging, AFAIK. "..ain't the same ballpark, it ain't the same league, it ain't even the same..." :-) Bullshit. They are both production quality compilers. People complain gcc bootstrap is too long. XLC at O2 (which is what they compile at, last i looked) isn't running the IPA middle end, etc. Again, "..ain't the same ballpark, it ain't the same league, it ain't even the same *** sport..." Software A provides X1 functionality. Software B provides X2 functionality. BTW, you know more, about XLC's SPEC numbers and GCC's SPEC numbers, than me. So in these X1 is not same as X2 but let's ignore that for moment and say X1 is almost same as X2. Software A and B is developed independently. They do not share source code. Software G compiles A in T1 time. Software X compiles B in T2 time. T1 is almost same as T2, so G is as fast as X. That's what you're trying to say? - Devang
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, Apr 28, 2005 at 11:03:51AM -0700, Matt Thomas wrote: > > Someone complained I was unfair in my gcc bootstrap times since > some builds included libjava/gfortran and some did not. > > So in the past day, I've done bootstrap with just c,c++,objc on > both 3.4 and gcc4.1. I've put the results in a web page at > http://3am-software.com/gcc-speed.html. The initial bootstrap > compiler was gcc3.3 and they are all running off the same base > of NetBSD 3.99.3. > > While taking out fortran and java reduced the disparity, there > is still a large increase in bootstrap times from 3.4 to 4.1. There's some new code in libstdc++ now (the TR1 stuff) that (last time I looked) takes a long time to build, and wasn't in 3.4. Could that be a factor? A comparison with just --enable-languages=c would eliminate any issues with libraries.
Re: GCC 4.1: Buildable on GHz machines only?
Matt Thomas <[EMAIL PROTECTED]> writes: > When I see the native stage2 m68k compiler spend 30+ minutes compute bound > with no paging activity compiling a single source file, I believe > that is an accurate term. Compiling stage3 on a 50MHz 68060 took 18 hours. > (That 30 minutes was for fold-const.c if you care to know). The fold-const.c compilation seems like a good specific candidate for a bug report at http://gcc.gnu.org/bugzilla/ If you can include the preprocessed file and profiler output from cc1 running on your system, there is a chance that this can be addressed. In general the gcc developers do a reasonable job of keeping the compiler from being too slow, but they do not use m68k systems, and they can only work on problems which people bring to their attention. And, yes, we clearly need to do something about the libjava build. Thanks. Ian
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 2005-04-28 at 11:08 -0700, Devang Patel wrote: > On Apr 28, 2005, at 10:54 AM, Daniel Berlin wrote: > > > On Thu, 2005-04-28 at 10:23 -0700, Devang Patel wrote: > >> On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote: > >> > >>> 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of > >>> yesterday. > >>> 2. Building XLC with (C,C++,Fortran) and a single backend takes > >>> roughly > >>> the same time as building GCC. And they aren't three staging, > >>> AFAIK. > >> > >> "..ain't the same ballpark, it ain't the same league, > >> it ain't even the same..." :-) > > > > Bullshit. > > They are both production quality compilers. > > > > People complain gcc bootstrap is too long. XLC at O2 (which is what > > they > > compile at, last i looked) isn't running the IPA middle end, etc. > > Again, > > "..ain't the same ballpark, it ain't the same league, > it ain't even the same *** sport..." > > Software A provides X1 functionality. > Software B provides X2 functionality. > This has nothing to do with it. We are talking about bootstrap times of compilers with roughly equivalent functionality. They both compile C, C++, and fortran. You can assume our backends and frontend are comparable in terms of functionality (they actually are, though not equivalent in performance generated by that functionality, but in time taken by each functional part they are). As i've said, we've already excluded the running time for the functionality we don't have (interprocedural middle end). --Dan
Re: GCC 4.1: Buildable on GHz machines only?
> Matt Thomas writes: Matt> So in the past day, I've done bootstrap with just c,c++,objc on Matt> both 3.4 and gcc4.1. Matt> While taking out fortran and java reduced the disparity, there Matt> is still a large increase in bootstrap times from 3.4 to 4.1. libstdc++ contains a lot more features in GCC 4.1, especially TR1. We understand that a lot of people use modern GCC on older hardware and the compilation speed can be frustrating. GCC supports a very diverse group of processors with greatly varying performance. Users also want new features and want to utilize newer hardware performance for more optimization in the same wall clock time. It is difficult, if not impossible, to satisfy everyone. The GCC developers are trying to improve compile time, but there are no magic bullets. If you provide testcases, developers do investigate ways to improve compile time when they have examples to test. It would be helpful if this discussion thread distinguished between compile time and bootstrap time. The two are related but not identical. While GCC compile time needs to improve, the amount of time that it takes to build GCC itself, relative to other commercial compilers with similar features and runtimes, is similar. David
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 28 Apr 2005, Ian Lance Taylor wrote: If you can include the preprocessed file and profiler output from cc1 running on your system, there is a chance that this can be addressed. GCC comes with a test suite and a means for submitting results. May I suggest that it might be useful to have a timing suite built similarly, so people can do this kind of thing really easily? Maybe someone can tell us if it is too big a job to create such a tool portably? I think having it running during stage3 would yield useful results for comparisons. A tool that came with GCC would ensure the data was in a useful format, and that tests were performed consistently. Hugh
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 2005-04-28 at 11:03 -0700, Matt Thomas wrote: > Someone complained I was unfair in my gcc bootstrap times since > some builds included libjava/gfortran and some did not. > > So in the past day, I've done bootstrap with just c,c++,objc on > both 3.4 and gcc4.1. I've put the results in a web page at > http://3am-software.com/gcc-speed.html. The initial bootstrap > compiler was gcc3.3 and they are all running off the same base > of NetBSD 3.99.3. > > While taking out fortran and java reduced the disparity, there > is still a large increase in bootstrap times from 3.4 to 4.1. I'm sure you used disable-checking on the 4.1 test, right? Since otherwise, it's a worthless comparison
Re: GCC 4.1: Buildable on GHz machines only?
On Thu, 28 Apr 2005, Daniel Berlin wrote: On Thu, 2005-04-28 at 11:03 -0700, Matt Thomas wrote: Someone complained I was unfair in my gcc bootstrap times since some builds included libjava/gfortran and some did not. So in the past day, I've done bootstrap with just c,c++,objc on both 3.4 and gcc4.1. I've put the results in a web page at http://3am-software.com/gcc-speed.html. The initial bootstrap compiler was gcc3.3 and they are all running off the same base of NetBSD 3.99.3. While taking out fortran and java reduced the disparity, there is still a large increase in bootstrap times from 3.4 to 4.1. I'm sure you used disable-checking on the 4.1 test, right? Since otherwise, it's a worthless comparison Forget it, i missed it in the light green on black text :)
Re: GCC 4.1: Buildable on GHz machines only?
Peter Barada wrote: The alternative of course is to do only crossbuilds. Is it reasonable to say that, for platforms where a bootstrap is no longer feasible, a successful crossbuild is an acceptable test procedure to use instead? A successful crossbuild is certainly the minimum concievable standard. Perhaps one should also require bootstrapping the C compiler alone; that would provide at least some sanity-checking. Unfortunately for some of the embedded targets(like the ColdFire V4e work I'm doing), a bootstrap is impossible due to limited memory and no usable mass-storage device on the hardware I have available, so hopefully a successful crossbuild will suffice. How about a successful crossbuild plus passing some regression test suite, e.g. gcc's, glibc's, and/or ltp's? Any one of them would provide a nice reality check. - Dan -- Trying to get a job as a c++ developer? See http://kegel.com/academy/getting-hired.html
Re: GCC 4.1: Buildable on GHz machines only?
Dan Nicolaescu <[EMAIL PROTECTED]> writes: > Matt Thomas <[EMAIL PROTECTED]> writes: > > > Richard Henderson wrote: > > > On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote: > > > > > >>I would expect it to be drastically faster. However this won't show up > > >>clearly in the bootstrap. The, bar none, longest bit of the bootstrap > > >>is building stage2; and stage1 is always built with optimization off and > > >>(IIRC) checking on. > > > > > > > > > Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when > > > building on risc machines. > > > > Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was > > I don't think that is enough, also edit gcc/Makefile.in and change the line: > STAGE1_CHECKING = -DENABLE_CHECKING -DENABLE_ASSERT_CHECKING > to be > STAGE1_CHECKING = > > Is there a better way to do this? STAGE1_CHECKING is not passed from > the toplevel make, so one cannot put it on the make bootstrap command > line... Following up on this to show some numbers. On a 2.8GHz P4, 1MB cache, 512MB RAM, Fedora Core 3 gcc HEAD configured with --enable-languages=c --disable-checking --disable-nls time make bootstrap > & out 1227.861u 53.623s 21:26.53 99.6%0+0k 0+0io 18pf+0w If gcc/Makefile.in is first edited as shown above, then the bootstrap time is: 983.769u 53.241s 17:33.12 98.4% 0+0k 0+0io 3573pf+0w A significant difference! Given that with --enable-languages=c the amount of stuff built with the slow stage1 compiler is minimal, the impact might be even higher when more languages are enabled (I haven't tried). It would be nice to have some way to disable STAGE1_CHECKING other than by editing gcc/Makefile.in (Sorry, I can't help with this, I don't know much about how the gcc configuration process).
Re: GCC 4.1: Buildable on GHz machines only?
>> Unfortunately for some of the embedded targets(like the ColdFire V4e >> work I'm doing), a bootstrap is impossible due to limited memory and >> no usable mass-storage device on the hardware I have available, so >> hopefully a successful crossbuild will suffice. > >How about a successful crossbuild plus >passing some regression test suite, >e.g. gcc's, glibc's, and/or ltp's? >Any one of them would provide a nice reality check. I'm open to running them if there's a *really* clear how-to to do it that takes into account remote hardware. -- Peter Barada [EMAIL PROTECTED]
Re: GCC 4.1: Buildable on GHz machines only?
On Apr 27, 2005, at 12:57 PM, Steven Bosscher wrote: Maybe the older platform should stick to the older compiler then, if it is too slow to support the kind of compiler that modern systems need. This is an unreasonable request. Consider NetBSD, which runs on new and old hardware. The OS continues to evolve, and that often requires adopting newer compilers (so e.g. other language features can be used in the base OS). The GCC performance issue is not new. It seems to come up every so often... last time I recall a discussion on the topic, it was thought that the new memory allocator (needed for pch) was cause cache-thrash (what was the resolution of that discussion, anyway?) -- thorpej
Re: GCC 4.1: Buildable on GHz machines only?
On Apr 27, 2005, at 7:41 AM, David Edelsohn wrote: GCC now supports C++, Fortran 90 and Java. Those languages have extensive, complicated runtimes. The GCC Java environment is becoming much more complete and standards compliant, which means adding more and more features. Except it's not just bootstrapping GCC. It's everything. When the NetBSD Project switched from 2.95.3 to 3.3, we had a noticeably increase in time to do the "daily" builds because the 3.3 compiler was so much slower at compiling the same OS source code. And we're talking almost entirely C code, here. -- thorpej
