Re: incremental linking?

[Hal Daume III]

Claus,

How did you get ghc to use gld when doing --make instead of standard
ld?  I'm having the exact same problem you were and I'd love to make it
work faster.

Thanks in advance,

 Hal

--
Hal Daume III

 Computer science is no more about computers| [EMAIL PROTECTED]
  than astronomy is about telescopes. -Dijkstra | www.isi.edu/~hdaume

On Sat, 30 Nov 2002, Claus Reinke wrote:

 
 It seems that Sun's ld was indeed the weak link. Switching to
 Gnu's ld (*) brought the linking time down to just under 1 minute,
 both on the machine that used to take 6 minutes and on the 
 older one that used to take 20 minutes!
 
 I don't know whether Sun's lds themselves are to blame or whether 
 ghc/gcc generate output that suits Gnu's ld better than it does Sun's 
 ld, but as long as ghc remains as it is, that doesn't really make a 
 difference for our purposes. 
 
 (slow) ld -V
 ld: Software Generation Utilities - Solaris Link Editors: 5.8-1.276
 
 (better) ld -V
 ld: Software Generation Utilities - Solaris Link Editors: 5.9-1.344
 
 (winner) ld -V
 GNU ld version 2.13.1
   Supported emulations:
elf32_sparc
elf64_sparc
 
 We haven't done extensive testing yet, and earlier versions
 of binutils (up to and including 2.13) are reported to have 
 problems on Solaris, so don't throw away Sun's tools, but it 
 looks as if our case is now closed (and incremental linking
 isn't an issue anymore with these new link times!-)
 
 Thanks for the helpful feedback, and Good Luck with the
 other suspiciously slow systems!
 
 Claus
 
 (*) a little stumbling block here: gcc refers to PATH only *after* 
   perusing its preconfigured search paths. In our case, those included 
   /usr/ccs/bin, so we had to set GCC_EXEC_PREFIX instead (which 
   is used before the preconfigured paths). Check with:
   gcc -print-search-dirs
 
 - Original Message - 
 From: Claus Reinke [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Sent: Friday, November 29, 2002 12:02 PM
 Subject: Re: incremental linking?
 
 
  I haven't been able to discern any pattern among those experiencing long
  link times so far, except that -export-dynamic flag used by the dynamic
  loader stuff seems to cause the linker to go off into space for a while.
  
  We're still investigating here, but just a quick summary for our own
  (large) project:
  
  - nfs doesn't seem to have too drastic effects, even in-memory disks
don't speed things up, time seems to be spend in computation
  - on our (admittedly overloaded and dated) main Sun Server, linking
 could take some 20 minutes!
  - we've found a more modern (and not yet well-utilized;-) Sun server,
bringing the time down to 6 minutes..:-(
  
  (from that, I thought linking might have to be expensive - how naive!-)
  
  - the same program on my rather old 366Mhz PII notebook links in
 about 1 minute (I didn't notice that at first, because overall compile
 time is longer on my notebook - but that turns out to be caused by 
 a single generated file, for which the assembler almost chokes; after
 all, the notebook only has 192Mb memory, and the disk is crammed)
  - with the laptop as reference, I'd guess the problem is not ghc's fault
 (unless it does things drastically different on cygwin vs solaris?)
  - on our Suns, gcc (and hence ghc) seem to use the native linker
  - sunsolve lists several linker patches to address problems like
linker orders of magnitude slower than Gnu's. We seem to have
those patches, but we're checking again..
  
  moral so far: if compilation of big projects takes a long time, it is worth
  checking where that time is spend. for the same project, on different
  systems, we've got different bottlenecks:
  
  - large (generated) files [all systems]: assembler needs an awful lot
of space (not enough space-compile takes forever)
  - network disks: import chasing takes a lot of time
  - Suns (?): linking takes too long
  
  will report again if we get better news..
  
  Claus
  
  PS. if we get linking times down to what seems possible, incremental
 linking would no longer be urgent - we'll see..
  
  
  ___
  Glasgow-haskell-users mailing list
  [EMAIL PROTECTED]
  http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
 
 ___
 Glasgow-haskell-users mailing list
 [EMAIL PROTECTED]
 http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
 

___
Glasgow-haskell-users mailing list
[EMAIL PROTECTED]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: incremental linking?

[Claus Reinke]

Hi Hal,

 How did you get ghc to use gld when doing --make instead of standard
 ld?  I'm having the exact same problem you were and I'd love to make it
 work faster.

[Simon] might be worth adding to the docs (title: linking times on Suns, 
Sun's ld vs GNU ld; explanation: see previous mails)?

ghc calls ld via gcc, so everything from the gcc man page about how gcc 
finds its tools seems to apply.

It could be as easy as setting PATH so that gnu's ld is found before sun's,
but as the message you quote said, the gcc installation might find the path
to sun's ld first by other routes.. 

  (*) a little stumbling block here: gcc refers to PATH only *after* 
perusing its preconfigured search paths. In our case, those included 
/usr/ccs/bin, so we had to set GCC_EXEC_PREFIX instead (which 
is used before the preconfigured paths).

So we had to do something like the following (assuming sh, and gnu's
binutils in /usr/local/packages):

GCC_EXEC_PREFIX=/usr/local/packages/binutils/bin/ ghc-5.04 --make ..

The handling of prefixes is explained for gcc's -B option in the gcc man
page (here the version for our suns):

 -Bprefix
 This option specifies where to find the executables,
 libraries, include files, and data files of the compiler
 itself.

 The compiler driver program runs one or more of the
 subprograms cpp, cc1, as and ld.  It tries prefix as a
 prefix for each program it tries to run, both with and
 without machine/version/.

 For each subprogram to be run, the compiler driver first
 tries the -B prefix, if any.  If that name is not found,
 or if -B was not specified, the driver tries two
 standard prefixes, which are /usr/lib/gcc/ and
 /usr/local/lib/gcc-lib/.  If neither of those results in
 a file name that is found, the unmodified program name
 is searched for using the directories specified in your
 PATH environment variable.

 The compiler will check to see if the path provided by
 the -B refers to a directory, and if necessary it will
 add a directory separator character at the end of the
 path.

 -B prefixes that effectively specify directory names
 also apply to libraries in the linker, because the
 compiler translates these options into -L options for
 the linker.  They also apply to includes files in the
 preprocessor, because the compiler translates these
 options into -isystem options for the preprocessor.  In
 this case, the compiler appends include to the prefix.

 The run-time support file libgcc.a can also be searched
 for using the -B prefix, if needed.  If it is not found
 there, the two standard prefixes above are tried, and
 that is all.  The file is left out of the link if it is
 not found by those means.

 Another way to specify a prefix much like the -B prefix
 is to use the environment variable GCC_EXEC_PREFIX.

 As a special kludge, if the path provided by -B is
 [dir/]stageN/, where N is a number in the range 0 to 9,
 then it will be replaced by [dir/]include.  This is to
 help with boot-strapping the compiler.

To see where gcc is looking, check the programs entry in the output of

gcc -print-search-dirs

and make sure your favourite ld is found first.

Cheers,
Claus

___
Glasgow-haskell-users mailing list
[EMAIL PROTECTED]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users