Re: making ./validate run tests on all CPUs by default
On Wed, 13 Aug 2014 11:39:56 +0200 Tuncer Ayaz wrote: > On Tue, Aug 12, 2014 at 10:31 PM, Sergei Trofimovich wrote: > > Good evening all! > > > > Currently when being ran './validate' script (without any parameters): > > - builds ghc using 2 parallel jobs > > - runs testsuite using 2 parallel jobs > > > > I propose to change the default value to amount of available CPUs: > > - build ghc using N+1 parallel jobs > > - run testsuite using N+1 parallel jobs > > > > Pros: > > - first-time users will get faster ./validate > > - seasoned users will need less tweaking for buildbots > > > > Cons: > > - for imbalanced boxes (32 cores, 8GB RAM) it might > >be quite painful to drag box out of swap > > > > What do you think about it? > > Isn't the memory use also a problem on boxes with a much lower > number of cores (e.g. 7.8 space leak(s))? > > On one machine with 1GB per core, I had to limit cabal install's > parallelism when using 7.8. It's true in general, but I would not expect such a massive growth on ghc source. Current -Rghc-timing shows ~300MBs per ghc process on amd64. The fallout examples are HsSyn and cabal's PackageDescription modules. ghc's build system is a bit different from Cabal's: - Cabal runs one 'ghc --make' instance for a whole package. It leads to massive RAM usage in case of a multitude of modules (highlighting-kate and qthaskell come to mind). - ghc's buld system uses one 'ghc -c' execution for a single .hs file (roughly) > Assuming the patch goes in, is there a way to limit parallel jobs > on the command line? The mechanism to set limit manually is the same as before: CPUS=8 ./validate It's the default that is proposed to be changed. -- Sergei signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: making ./validate run tests on all CPUs by default
On Tue, Aug 12, 2014 at 10:31 PM, Sergei Trofimovich wrote: > Good evening all! > > Currently when being ran './validate' script (without any parameters): > - builds ghc using 2 parallel jobs > - runs testsuite using 2 parallel jobs > > I propose to change the default value to amount of available CPUs: > - build ghc using N+1 parallel jobs > - run testsuite using N+1 parallel jobs > > Pros: > - first-time users will get faster ./validate > - seasoned users will need less tweaking for buildbots > > Cons: > - for imbalanced boxes (32 cores, 8GB RAM) it might >be quite painful to drag box out of swap > > What do you think about it? Isn't the memory use also a problem on boxes with a much lower number of cores (e.g. 7.8 space leak(s))? On one machine with 1GB per core, I had to limit cabal install's parallelism when using 7.8. Assuming the patch goes in, is there a way to limit parallel jobs on the command line? ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
making ./validate run tests on all CPUs by default
Good evening all! Currently when being ran './validate' script (without any parameters): - builds ghc using 2 parallel jobs - runs testsuite using 2 parallel jobs I propose to change the default value to amount of available CPUs: - build ghc using N+1 parallel jobs - run testsuite using N+1 parallel jobs Pros: - first-time users will get faster ./validate - seasoned users will need less tweaking for buildbots Cons: - for imbalanced boxes (32 cores, 8GB RAM) it might be quite painful to drag box out of swap What do you think about it? Actual patch: https://phabricator.haskell.org/D146 Thanks! -- Sergei signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
