Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
--+-
  Reporter:  nr   |  Owner:  
  Type:  feature request  | Status:  closed  
  Priority:  normal   |  Milestone:  6.12.1  
 Component:  Build System |Version:  6.11
Resolution:  wontfix  |   Keywords:  
Difficulty:  Unknown  | Os:  Unknown/Multiple
  Testcase:   |   Architecture:  Unknown/Multiple
   Failure:  None/Unknown |  
--+-
Changes (by igloo):

  * status:  new => closed
  * failure:  => None/Unknown
  * resolution:  => wontfix

Comment:

 I don't think there's a cure for this that isn't worse than the disease,
 so I'm closing this as wont-fix.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  6.12.1  
   Component:  Build System  |  Version:  6.11
Severity:  minor |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Changes (by igloo):

  * milestone:  => 6.12.1

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  minor |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by simonmar):

 Replying to [comment:10 nr]:
 > Replying to [comment:9 simonmar]:
 >
 >   Surely most of the time the dependency file will be there anyway?
 >
 > Actually no.  The problem is that I'm propagating a change in the
 representation of low-level intermediate code, and it will be a very long
 time before all the modules in the compiler are once again mutually
 consistent.  And apparently as long as any module does not compile, GHC
 cannot generate the dependency file.  I hope I am wrong about this and am
 just doing something stupid.  Please say I am?

 If I touch a source file in the compiler and then say `make 1` (in
 `compiler/`), I don't see the error message.  Even if I just say `make` in
 `compiler`, which rebuilds the dependencies if anything is out of date, I
 still don't see the error message.

 If you start from a position where you don't have the dependency file,
 then you do need to get to a state where all the imports exist in order to
 build the dependencies (the dependency-generator parses each source file
 up the end of the imports, so errors further down won't affect it).

 > If I'm not doing something stupid, can you suggest where I might go to
 filter the output of make in my own private world?

 In the top-level `Makefile`, something like this:
 {{{
 @echo "===--- finished updating makefiles"
 $(MAKE) -r --no-print-directory -f ghc.mk $@  | sed 'd/.depend: No
 such file or directory/'
 }}}

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  minor |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by nr):

 Replying to [comment:9 simonmar]:

 I certainly didn't mean to suggest that you even ''consider'' another
 attack on the build system!  I really was thinking about a student
 ''here'' :-)

 I'm interested in nmake exactly because relative to gmake or standard
 make, nmake definitely requires fewer phase distinctions, and it may
 eliminate phase distinctions altogether.  No phase distinctions -> no
 phase ordering -> speed.  But it sounds like the hacks you've put in will
 help a lot.

   Surely most of the time the dependency file will be there anyway?

 Actually no.  The problem is that I'm propagating a change in the
 representation of low-level intermediate code, and it will be a very long
 time before all the modules in the compiler are once again mutually
 consistent.  And apparently as long as any module does not compile, GHC
 cannot generate the dependency file.  I hope I am wrong about this and am
 just doing something stupid.  Please say I am?

 If I'm not doing something stupid, can you suggest where I might go to
 filter the output of make in my own private world?

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  minor |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by simonmar):

 {{{
 Tue May 19 01:33:50 PDT 2009  Simon Marlow 
   * allow phases to be omitted by setting OMIT_PHASE_[123]=YES

 Tue May 19 01:34:19 PDT 2009  Simon Marlow 
   * make [123] omits phases 1,2, and 3
 }}}

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  minor |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by simonmar):

 Replying to [comment:7 nr]:
 > Two comments:
 >
 >   * Regarding the speed of the build, I think it is really worth
 exploring Glenn Fowler's nmake.  nmake is engineered by real engineers to
 be portable and to work at scale.  It is entirely possible that it might
 make the safe path also fast.  It could even make it possible (although
 time-consuming and tedious) to eliminate the separate 'configure' step.
 What do you think of this as a possible student project?

 The speed of `make` is not the limiting factor.  IIRC, at the end of a
 complete GHC build the single `make` process has used only a couple of
 seconds of CPU time.

 nmake may well be better than GNU make (though it's not clear that it does
 everything we need).  Anyway I don't have the energy to rewrite the build
 system again right now - maybe in a few years!

 The real problem we would like to solve is the way GNU make handles
 rebuilding included makefiles, which leads to the need for the
 [wiki:Building/Architecture/Idiom/PhaseOrdering phase ordering].  This
 adds 4 seconds or so to each rebuild, not to mention complexity and
 fragility in the build system.  Now, two of those phases are only needed
 because GHCs before 6.11 didn't generate makefile dependencies between
 packages in the way the new build system needs, so in the future when
 bootstrapping with 6.11+ we can drop the two slowest phases, which should
 help a lot.

 For now I've made `make 1` and `make 2` omit phases 1-3.  It's much
 quicker for me now.

 >   * Having established the safe but slow path, is there a way to make it
 less noisy?  In particular, is there a way to change ghc/ghc.mk so that it
 does ''not'' bleat if the dependency file is missing?  I have upgraded
 severity to 'minor' (from 'trivial') because this error message interrupts
 my workflow in emacs ({{{C-x `}}} now has to stop at an extra error
 message... sometimes).

 I agree the message is annoying, but I can't see an easy way to quiet it
 short of filtering the output of make (which is possible, but exceedingly
 ugly).  Surely most of the time the dependency file will be there anyway?

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  minor |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Changes (by nr):

  * severity:  trivial => minor

Comment:

 Two comments:

   * Regarding the speed of the build, I think it is really worth exploring
 Glenn Fowler's nmake.  nmake is engineered by real engineers to be
 portable and to work at scale.  It is entirely possible that it might make
 the safe path also fast.  It could even make it possible (although time-
 consuming and tedious) to eliminate the separate 'configure' step.  What
 do you think of this as a possible student project?

   * Having established the safe but slow path, is there a way to make it
 less noisy?  In particular, is there a way to change ghc/ghc.mk so that it
 does ''not'' bleat if the dependency file is missing?  I have upgraded
 severity to 'minor' (from 'trivial') because this error message interrupts
 my workflow in emacs ({{{C-x `}}} now has to stop at an extra error
 message... sometimes).

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  trivial   |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by simonmar):

 Norman - welcome to the world of non-recursive make :-)  Even when you say
 `make` in `compiler/`, the build system is bringing every dependency of
 GHC up to date in order to start building GHC itself.  Of course this
 might well be overkill, but the design principle was "safety first": we
 want to ensure that the tree never ends up in an inconsistent state, at
 least unless the user explicitly says they want to play fast and loose.

 What we should do now, having established the safe but slow path, is to
 identify exactly the places where we want to omit dependencies in order to
 make development a little less painful.  So thanks for starting the
 discussion.

 `make 1` is an example of one such target: it prevents the dependencies of
 GHC being rebuilt.  I think what you're additionally worried about is the
 "updating makefiles..." stages, which are also about consistency.  If you
 want, you can cheat and omit these too; e.g. at the top level

 {{{
 $ make -f ghc.mk all_ghc_stage1 compiler_stage1_NO_BUILD_DEPS=YES
 }}}

 this should be the fastest way to rebuild stage1.  Perhaps this is
 something we should enshrine as a rule, or perhaps this should be what
 happens when you say `make 1`?  I have some concerns about what could
 happen if you have touched something else in the tree - weird and wacky
 failures could ensue because we're not bringing the included makefiles up
 to date in the right order, but hopefully this wouldn't permanently kill
 your tree, and a top-level `make` should be able to recover.  Let me know
 your experiences anyway.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  trivial   |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by nr):

 I'm afraid
 {{{
 make 1
 }}}
 suffers from the same problem:
 {{{
 : n...@labrador 3648 [branch=norman] ; make 1
 make -C .. stage=1 all_ghc_stage1 compiler_stage1_NO_BUILD_DEPS=YES
 make[1]: Entering directory `/usr/local/nr/git/ghc/experimental'
 ===--- updating makefiles phase 0
 make -r --no-print-directory -f ghc.mk phase=0 just-makefiles
 ===--- updating makefiles phase 1
 make -r --no-print-directory -f ghc.mk phase=1 just-makefiles
 ===--- updating makefiles phase 2
 make -r --no-print-directory -f ghc.mk phase=2 just-makefiles
 ===--- updating makefiles phase 3
 make -r --no-print-directory -f ghc.mk phase=3 just-makefiles
 ghc/ghc.mk:90: ghc/stage1/build/.depend: No such file or directory
 /usr/bin/ghc  -H64m -O0 -fasm  -package-conf libraries/bootstrapping.conf
 -package-name ghc-6.11.20090513 -hide-all-packages -i -icompiler/nativeGen
 -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn
 -icompiler/cprAnalysis -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn
 -icompiler/iface -icompiler/main -icompiler/parser -icompiler/prelude
 -icompiler/profiling -icompiler/rename -icompiler/simplCore
 -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn
 -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils
 -icompiler/vectorise -icompiler/stage1/build
 -icompiler/stage1/build/autogen -Icompiler/stage1/build
 -Icompiler/stage1/build/autogen -Icompiler/stage1
 -Icompiler/../libraries/base/cbits -Icompiler/../libraries/base/include
 -Icompiler/. -Icompiler/parser -Icompiler/utils   -optP-include
 -optPcompiler/stage1/build/autogen/cabal_macros.h -package Cabal-1.7.0
 -package array-0.2.0.0 -package base-4.0.0.0 -package bytestring-0.9.1.4
 -package containers-0.2.0.0 -package directory-1.0.0.2 -package
 filepath-1.1.0.1 -package haskell98-1.0.1.0 -package hpc-0.5.0.2 -package
 old-time-1.0.0.1 -package process-1.0.1.0 -package unix-2.3.1.0  -#include
 cutils.h -DSTAGE=1 -O -fasm -Wall -fno-warn-name-shadowing -fno-warn-
 orphans -XCPP -XMagicHash -XUnboxedTuples -XPatternGuards
 -XForeignFunctionInterface -XEmptyDataDecls -XTypeSynonymInstances
 -XMultiParamTypeClasses -XFlexibleInstances -XRank2Types
 -XScopedTypeVariables -XDeriveDataTypeable -XRelaxedPolyRec-odir
 compiler/stage1/build -hidir compiler/stage1/build -stubdir
 compiler/stage1/build -hisuf hi -osuf  o -hcsuf hc -c
 compiler/codeGen/StgCmmProf.hs -o compiler/stage1/build/StgCmmProf.o

 compiler/codeGen/StgCmmProf.hs:350:41:
 `CmmAGraph' is not applied to enough type arguments
 Expected kind `*', but `CmmAGraph' has kind `* -> * -> *'
 In the type `FCode CmmAGraph'
 In the type `CollectedCCs -> FCode CmmAGraph'
 In the type signature for `initCostCentres':
   initCostCentres :: CollectedCCs -> FCode CmmAGraph

 compiler/codeGen/StgCmmProf.hs:417:30:
 `CmmAGraph' is not applied to enough type arguments
 Expected kind `?', but `CmmAGraph' has kind `* -> * -> *'
 In the type `CostCentre -> CmmAGraph'
 In the type signature for `mkRegisterCC':
   mkRegisterCC :: CostCentre -> CmmAGraph

 compiler/codeGen/StgCmmProf.hs:435:36:
 `CmmAGraph' is not applied to enough type arguments
 Expected kind `?', but `CmmAGraph' has kind `* -> * -> *'
 In the type `CostCentreStack -> CmmAGraph'
 In the type signature for `mkRegisterCCS':
   mkRegisterCCS :: CostCentreStack -> CmmAGraph

 compiler/codeGen/StgCmmProf.hs:479:27:
 `CmmAGraph' is not applied to enough type arguments
 Expected kind `?', but `CmmAGraph' has kind `* -> * -> *'
 In the type `CmmExpr -> CmmAGraph'
 In the type signature for `bumpSccCount':
   bumpSccCount :: CmmExpr -> CmmAGraph
 make[2]: *** [compiler/stage1/build/StgCmmProf.o] Error 1
 make[1]: *** [all_ghc_stage1] Error 2
 make[1]: Leaving directory `/usr/local/nr/git/ghc/experimental'
 make: *** [1] Error 2
 }}}
 I may have a go at the doco once I understand a little better what's going
 on.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  trivial   |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by simonpj):

 Try
 {{{
 make 1
 }}}

 
http://hackage.haskell.org/trac/ghc/wiki/Building/Using#RebuildingtheGHCbinaryaftermakingchanges

 Does that do it?  If the above documentation isn't clear enough, or isn't
 easy enough to find, could you modify it so that it is?  It's a wiki.
 Thanks.

 Simon

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  trivial   |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Comment (by nr):

 I just pulled from the darcs head a few days ago, so I presume I'm
 building 6.11.

 I don't know where {{{Makefile-stage1}}} comes from---maybe it is a stale
 version from a while back---but it is very, very useful.  Comparing
 {{{
 make -f Makefile dist/stage1/ZipGF.o
 }}}
 and
 {{{
 make -f Makefile-stage1 dist/stage1/ZipGF.o
 }}}
 the second is considerably faster.

 I basically am asking for a fast path to attempt to compile a .o file from
 possibly stale dependencies.  I am very, very eager to trade accuracy for
 speed here.  If something strange happens I can always resort to the full
 Makefile.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|Owner:  
Type:  feature request   |   Status:  new 
Priority:  normal|Milestone:  
   Component:  Build System  |  Version:  6.11
Severity:  trivial   |   Resolution:  
Keywords:|   Difficulty:  Unknown 
Testcase:|   Os:  Unknown/Multiple
Architecture:  Unknown/Multiple  |  
-+--
Changes (by simonmar):

  * difficulty:  => Unknown

Comment:

 `Makefile-stage1` doesn't exist any more - what GHC version are you
 building here?

 The missing `.depend` file errors are annoying, but unfortunately not easy
 to silence.  I think I remember coming across a trick somewhere for
 shutting them up, but can't remember where it was now.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

Re: [GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
--+-
 Reporter:  nr|  Owner:  
 Type:  feature request   | Status:  new 
 Priority:  normal|  Milestone:  
Component:  Build System  |Version:  6.11
 Severity:  trivial   | Resolution:  
 Keywords:|   Testcase:  
   Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
--+-
Changes (by nr):

  * severity:  normal => trivial

Comment:

 It turns out that by going directly to {{{Makefile-stage1}}} I can avoid
 this error message and also a powerful lot of futzing around.  I probably
 lose some dependencies but it is worth it for the speed.

 I've knocked the severity down as low as possible; if someone wants to
 mark it wontfix, I won't bleat.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs

[GHC] #3226: please eliminate error message from Make on every build

[GHC]

#3226: please eliminate error message from Make on every build
-+--
Reporter:  nr|  Owner:  
Type:  feature request   | Status:  new 
Priority:  normal|  Component:  Build System
 Version:  6.11  |   Severity:  normal  
Keywords:|   Testcase:  
  Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
-+--
 If I attempt to build a .o file (which is my normal workflow), I
 ''always'' get this error message:
 {{{
 ghc/ghc.mk:90: ghc/stage1/build/.depend: No such file or directory
 }}}
 If this outcome is expected (is it an early-phase thing?), it would be
 very, very nice to have stderr redirected.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs