The libtool folks seem to be making some progress in attacking our problems. I had forwarded them Richard's data on the libtool performance problems.
When they have something ready to try, I hope someone on our end who knows about this stuff (Alexandre? Java folks?) can try it out. I'd send a URL to the thread but the lists.gnu.org site seems to be down; if it ever comes back there is http://lists.gnu.org/archive/html/bug-libtool/ ----- Forwarded message from Ralf Wildenhues <[EMAIL PROTECTED]> ----- Date: Thu, 5 May 2005 12:11:29 +0200 From: Ralf Wildenhues <[EMAIL PROTECTED]> To: Joe Buck <[EMAIL PROTECTED]>, bug-libtool@gnu.org Cc: libtool-patches@gnu.org Subject: Re: [EMAIL PROTECTED]: libjava build times] Mail-Followup-To: Joe Buck <[EMAIL PROTECTED]>, bug-libtool@gnu.org, libtool-patches@gnu.org In-Reply-To: <[EMAIL PROTECTED]> Organization: Department of Numerical Simulation, University of Bonn X-pstn-levels: (S:99.90000/99.90000 R:95.9108 P:95.9108 M:97.0232 C:98.7678 ) X-pstn-settings: 2 (0.5000:0.5000) s gt3 gt2 gt1 r p m c X-pstn-addresses: from <[EMAIL PROTECTED]> [27/2] X-Bogosity: No Hi Joe, others, * Ralf Wildenhues wrote on Wed, May 04, 2005 at 09:40:21PM CEST: > * Ralf Wildenhues wrote on Wed, May 04, 2005 at 06:30:57PM CEST: > > * Joe Buck wrote on Mon, May 02, 2005 at 08:01:49PM CEST: > > > > > > We really need something done about this problem, as it interferes > > > with our ability to efficiently develop GCC. Thank you for providing this example! It shows the bottlenecks neatly. I will tackle 'link mode' first, as I have worked on that previously and know roughly what to do. libtool-cache integration comes afterwards (and probably in form of a small compiled program). > > How many objects does libjava contain? Rather 100 or 1000? Do you > > need relinking because of command line length? > > I am compiling libjava ATM to see what happens. OK, step 1: Executing the GCC provided libtool script for libgcj0_convenience.la (warm cache) takes 294.35user 45.47system 6:13.42elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+6043907minor)pagefaults 0swaps on my laptop (Pentium M 1.8), with --dry-run, so without doing any real work. One significant bugger is the quadratic renaming loop, and all the stuff that I already fixed in Libtool HEAD. The attached rough patch kills the loop and brings Libtool HEAD down to 170.22user 10.41system 3:17.38elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+1483544minor)pagefaults 0swaps building the piecewise archiving cmdline better gets me 133.80user 4.28system 2:36.78elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (4major+747642minor)pagefaults 0swaps I'm pretty sure I can get it quite a bit faster even. The patches need cleanup so that they use allowed file names and work properly in corner cases as well, but those don't scale with the number of objects, so they matter less. If you are interested in further work: followup patches and cleaned up versions of these will be posted to the libtool-patches list only. Regards, Ralf * config/ltmain.m4sh (func_mode_link): Kill quadratic loop while renaming objects. Reported by Joe Buck <[EMAIL PROTECTED]>. Index: config/ltmain.m4sh =================================================================== RCS file: /cvsroot/libtool/libtool/config/ltmain.m4sh,v retrieving revision 1.66 diff -u -r1.66 ltmain.m4sh --- config/ltmain.m4sh 27 Apr 2005 11:30:38 -0000 1.66 +++ config/ltmain.m4sh 5 May 2005 09:42:10 -0000 @@ -6298,43 +6298,31 @@ # static archive out of a convenience library, or when linking # the entirety of a libtool archive into another (currently # not supported by libtool). - if (for obj in $oldobjs - do - func_basename "$obj" - $ECHO "$func_basename_result" - done | sort | sort -uc >/dev/null 2>&1); then - : - else + gentop="$output_objdir/${outputname}x" + generated="$generated $gentop" + func_mkdir_p "$gentop" + objs_bnames="_objs_basenames" + objs_nonuniq="_objs_nonuniq" + $ECHO "$oldobjs" | $SP2NL | $SED -n -e "$basename" -e '/./p' >"$objs_bnames" + sort "$objs_bnames" | uniq -d > "$objs_nonuniq" + if test -s "$objs_nonuniq" + then $ECHO "copying selected object files to avoid basename conflicts..." - gentop="$output_objdir/${outputname}x" - generated="$generated $gentop" - func_mkdir_p "$gentop" save_oldobjs=$oldobjs - oldobjs= - counter=1 - for obj in $save_oldobjs + : >_replace_script + # make sure we use unique names + counter=`$SED -n '/^lt-[[0-9]][[0-9]]*/ { s/^lt-//; s/[[^0-9]].*//; p; }' _objs_basenames \ + | sort -n | $SED -n '$p'` + $ECHO "$oldobjs" | $SP2NL | $SED -n -e '/./p' >_objs + paste "$objs_bnames" _objs | sort >_objs_plus_bnames + join "$objs_nonuniq" _objs_plus_bnames | while read objbase obj do - func_basename "$obj" - objbase="$func_basename_result" - case " $oldobjs " in - " ") oldobjs=$obj ;; - *[[\ /]]"$objbase "*) - while :; do - # Make sure we don't pick an alternate name that also - # overlaps. - newobj=lt$counter-$objbase - counter=`expr $counter + 1` - case " $oldobjs " in - *[[\ /]]"$newobj "*) ;; - *) if test ! -f "$gentop/$newobj"; then break; fi ;; - esac - done - func_show_eval "ln $obj $gentop/$newobj || cp $obj $gentop/$newobj" - oldobjs="$oldobjs $gentop/$newobj" - ;; - *) oldobjs="$oldobjs $obj" ;; - esac + counter=`expr $counter + 1` + newobj=lt$counter-$objbase + func_show_eval "ln $obj $gentop/$newobj || cp $obj $gentop/$newobj" + $ECHO "s#^$obj\$#$gentop/$newobj#" >>_replace_script done + oldobjs=`$SED -f _replace_script _objs | $NL2SP` fi eval cmds=\"$old_archive_cmds\" ----- End forwarded message -----
