Building GHC for performance testing
Hi, I'd like to set up a performance build bot for GHC, but before I can do that I need a script that reliably builds GHC and runs nofib. Do we have such a script? Here's a strawman proposal for one: cabal install happy alex git clone git://git.haskell.org/ghc.git cd ghc ./sync-all --nofib get perl boot ./configure make cd nofib make clean make boot make -k mode=slow Questions: * Does this look sensible? * Is there a way to only build and run a subset of the benchmarks? * Are there any tweaks to mk/build.mk we can do to make the build faster without compromising the results? * Is there a way to do this in a cheap throwaway VM like travis-ci does? Could such a VM already provide GHC and the required libs to make the whole thing hermetic? ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Building GHC for performance testing
On 2014-03-06 at 10:50:09 +0100, Johan Tibell wrote: I'd like to set up a performance build bot for GHC, but before I can do that I need a script that reliably builds GHC and runs nofib. Do we have such a script? Here's a strawman proposal for one: cabal install happy alex git clone git://git.haskell.org/ghc.git cd ghc ./sync-all --nofib get perl boot ./configure make cd nofib make clean make boot make -k mode=slow Questions: * Does this look sensible? If the script is supposed to start in an empty environment, then it looks good to me * Is there a way to only build and run a subset of the benchmarks? * Are there any tweaks to mk/build.mk we can do to make the build faster without compromising the results? You should disable LaTeX generation, and maybe also haddock generation; and possibly remove the libraries/dph folder (unless that's becomes part of the benchmarks) how often would you run the benchmarks? once a day? * Is there a way to do this in a cheap throwaway VM like travis-ci does? Could such a VM already provide GHC and the required libs to make the whole thing hermetic? Btw, does benchmarking inside a VM provide good enough measurements? E.g. I'd be worried from noise due to vSMP scheduling (as GHC's GC is rather sensible to proper SMP scheduling) and maybe from noise coming from other VMs trashing the caches. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Building GHC for performance testing
On 06/03/14 09:50, Johan Tibell wrote: Hi, I'd like to set up a performance build bot for GHC, but before I can do that I need a script that reliably builds GHC and runs nofib. Do we have such a script? Here's a strawman proposal for one: cabal install happy alex git clone git://git.haskell.org/ghc.git cd ghc ./sync-all --nofib get perl boot ./configure make cd nofib make clean make boot make -k mode=slow Questions: * Does this look sensible? * Is there a way to only build and run a subset of the benchmarks? * Are there any tweaks to mk/build.mk we can do to make the build faster without compromising the results? * Is there a way to do this in a cheap throwaway VM like travis-ci does? Could such a VM already provide GHC and the required libs to make the whole thing hermetic? ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs I think this ties in with the whole issue of not having nightlies/test bots running. I think if the infrastructure for that was in place then this would simply be a part of the functionality. re: subset of benchmarks, I believe they are just regular tests, in which case you can explicitly pass in which test cases should be ran. Correct me if I'm wrong. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Strange file
What is this file? nofib/real/HMMS/lib/haskell/Builtin.hi It's a mystery! Can we delete it? Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
[PATCH] time library fix for PowerPC
Please consider this as a pull request. https://github.com/ggreif/packages-time/commit/83f08b8896b950707a910fed4e30bddb7ee7f52f It fixes an ugly code size blowup that trips the PowerPC assembler (NCG mode). Cheers, Gabor ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Strange file
On Thu, Mar 6, 2014 at 1:06 PM, Simon Peyton Jones simo...@microsoft.comwrote: What is this file? nofib/real/HMMS/lib/haskell/Builtin.hi It's a mystery! Can we delete it? I think it's safe to assume that it was added by mistake. David even tried to remove it once: tmp/ghc/nofib$ git log real/HMMS/lib/haskell/Builtin.hi commit af36a39eed2e598c1f4c49dd726d3732b53b0f46 Author: David Terei davidte...@gmail.com Date: Thu Jan 19 12:29:04 2012 -0800 Revert Move benchmarks into benchmark/ subdir. This reverts commit 0449cb065437fc8014b6669e5f1c2c8f4a926d16. Conflicts: .gitignore commit 0449cb065437fc8014b6669e5f1c2c8f4a926d16 Author: David Terei davidte...@gmail.com Date: Tue Jan 17 10:53:12 2012 -0800 Move benchmarks into benchmark/ subdir. commit 0bd88c95b14252969c9dd0a5eaa9da219af5f85f Author: partain unknown Date: Mon Jan 8 20:13:28 1996 + [project @ 1996-01-08 20:13:28 by partain] Initial revision ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Building GHC for performance testing
There's also the issue that certain vms disable some simd instructions(I think), and while we don't have many libs that are simd heavy as yet this could be an issue in certain future benchmarks? (I could be completely wrong though. ) On Thursday, March 6, 2014, Herbert Valerio Riedel hvrie...@gmail.com wrote: On 2014-03-06 at 10:50:09 +0100, Johan Tibell wrote: I'd like to set up a performance build bot for GHC, but before I can do that I need a script that reliably builds GHC and runs nofib. Do we have such a script? Here's a strawman proposal for one: cabal install happy alex git clone git://git.haskell.org/ghc.git cd ghc ./sync-all --nofib get perl boot ./configure make cd nofib make clean make boot make -k mode=slow Questions: * Does this look sensible? If the script is supposed to start in an empty environment, then it looks good to me * Is there a way to only build and run a subset of the benchmarks? * Are there any tweaks to mk/build.mk we can do to make the build faster without compromising the results? You should disable LaTeX generation, and maybe also haddock generation; and possibly remove the libraries/dph folder (unless that's becomes part of the benchmarks) how often would you run the benchmarks? once a day? * Is there a way to do this in a cheap throwaway VM like travis-ci does? Could such a VM already provide GHC and the required libs to make the whole thing hermetic? Btw, does benchmarking inside a VM provide good enough measurements? E.g. I'd be worried from noise due to vSMP scheduling (as GHC's GC is rather sensible to proper SMP scheduling) and maybe from noise coming from other VMs trashing the caches. ___ ghc-devs mailing list ghc-devs@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Standalone Deriving, GND and roles in 7.8.1-rc2
This has now been entered as bug #8851. Track progress there if you're
interested.
Thanks for bringing this up!
Richard
On Mar 5, 2014, at 4:34 PM, Edward Kmett ekm...@gmail.com wrote:
As usual I'm the source of more headaches for Richard. ;)
-Edward
On Wed, Mar 5, 2014 at 3:57 PM, Sergei Trofimovich sly...@gmail.com wrote:
On Wed, 5 Mar 2014 15:40:27 -0500
Richard Eisenberg e...@cis.upenn.edu wrote:
I think those should be the same and that you've discovered a bug. What's
the definition of the Parsing class?
Here it goes [1]:
class Alternative m = Parsing m where
notFollowedBy :: (Monad m, Show a) = m a - m ()
notFollowedBy p = try ((try p = unexpected . show) | pure ())
{-# INLINE notFollowedBy #-}
[1]:
https://github.com/ekmett/parsers/blob/master/src/Text/Parser/Combinators.hs#L230
--
Sergei
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: [PATCH] time library fix for PowerPC
Rejected. The time library git repo only takes converted patches from its darcs upstream, see https://ghc.haskell.org/trac/ghc/wiki/Repositories/Upstream. However, I can see about making the equivalent change in http://code.haskell.org/time/, or you can send me a darcs patch. -- Ashley On 2014-03-06 04:29, Gabor Greif wrote: Please consider this as a pull request. https://github.com/ggreif/packages-time/commit/83f08b8896b950707a910fed4e30bddb7ee7f52f It fixes an ugly code size blowup that trips the PowerPC assembler (NCG mode). Cheers, Gabor ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Where's the implementation of newArray#
Is the below code it (from rts/PrimOps.cmm)? Does it being in PrimOps.cmm
mean that it's always out-of-line?
stg_newArrayzh ( W_ n /* words */, gcptr init )
{
W_ words, size;
gcptr p, arr;
again: MAYBE_GC(again);
// the mark area contains one byte for each 2^MUT_ARR_PTRS_CARD_BITS
words
// in the array, making sure we round up, and then rounding up to a
whole
// number of words.
size = n + mutArrPtrsCardWords(n);
words = BYTES_TO_WDS(SIZEOF_StgMutArrPtrs) + size;
(ptr arr) = ccall allocate(MyCapability() ptr,words);
TICK_ALLOC_PRIM(SIZEOF_StgMutArrPtrs, WDS(n), 0);
SET_HDR(arr, stg_MUT_ARR_PTRS_DIRTY_info, CCCS);
StgMutArrPtrs_ptrs(arr) = n;
StgMutArrPtrs_size(arr) = size;
// Initialise all elements of the the array with the value in R2
p = arr + SIZEOF_StgMutArrPtrs;
for:
if (p arr + WDS(words)) {
W_[p] = init;
p = p + WDS(1);
goto for;
}
// Initialise the mark bits with 0
for2:
if (p arr + WDS(size)) {
W_[p] = 0;
p = p + WDS(1);
goto for2;
}
return (arr);
}
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
new Windows build problem
Sigh. Totally new problem with building GHC (HEAD) on Windows.. inplace/bin/ghc-stage2.exe -hisuf hi -osuf o -hcsuf hc -static -H32m -O -Werror -Wall -H64m -O0-package-name haskell98-2.0.0.3 -hide-all-packages -i -ilibraries/haskell98/. -ilibraries/haskell98/dist-install/build -ilibraries/haskell98/dist-install/build/autogen -Ilibraries/haskell98/dist-install/build -Ilibraries/haskell98/dist-install/build/autogen -Ilibraries/haskell98/. -optP-include -optPlibraries/haskell98/dist-install/build/autogen/cabal_macros.h -package array-0.5.0.0 -package base-4.7.0.0 -package directory-1.2.0.2 -package old-locale-1.0.0.6 -package old-time-1.1.0.2 -package process-1.2.0.0 -package time-1.4.2 -Wall -XHaskell98 -O2 -O -dcore-lint -fno-warn-amp -fno-warn-deprecated-flags -dcore-lint -no-user-package-db -rtsopts -odir libraries/haskell98/dist-install/build -hidir libraries/haskell98/dist-install/build -stubdir libraries/haskell98/dist-install/build -c libraries/haskell98/./Directory.hs -o libraries/haskell98/dist-install/build/Directory.o bash: [6536: 1 (255)] tcsetattr: Inappropriate ioctl for device HEAD $ What on earth is this ioctl stuff? The above is repeatable; same command line, same result. But I'm also getting random seg-faults when building libraries with the stage2 compiler libraries/random/ghc.mk:5: recipe for target 'libraries/random/dist-install/build/System/Random.o' failed make[1]: *** [libraries/random/dist-install/build/System/Random.o] Segmentation fault This is NOT repeatable. Saying 'make' again gets further. Any ideas? Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Building GHC for performance testing
Hello Johan, Off the top of my head: Technically speaking, the compiler does not have to be optimized to do build tests; just the libraries. So you could probably make the builds run a bit faster this way. Your command for nofib will only run the standard benchmarks; to run, for example, the GC benchmarks, you will have to cd into the gc directory and run make there. You may also need to vary the tests parameters in other ways, such as threaded and un-threaded. Cheers, Edward Excerpts from Johan Tibell's message of 2014-03-06 01:50:09 -0800: Hi, I'd like to set up a performance build bot for GHC, but before I can do that I need a script that reliably builds GHC and runs nofib. Do we have such a script? Here's a strawman proposal for one: cabal install happy alex git clone git://git.haskell.org/ghc.git cd ghc ./sync-all --nofib get perl boot ./configure make cd nofib make clean make boot make -k mode=slow Questions: * Does this look sensible? * Is there a way to only build and run a subset of the benchmarks? * Are there any tweaks to mk/build.mk we can do to make the build faster without compromising the results? * Is there a way to do this in a cheap throwaway VM like travis-ci does? Could such a VM already provide GHC and the required libs to make the whole thing hermetic? ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
RE: new Windows build problem
Actually, the two problems seem to be the same. A fresh build from scratch
yields
libraries/random/ghc.mk:5: recipe for target
'libraries/random/dist-install/build/System/Random.o' failed
make[1]: *** [libraries/random/dist-install/build/System/Random.o] Segmentation
fault
Then sniping the command line that tried to build Random.o yields
inplace/bin/ghc-stage2.exe -hisuf hi -osuf o -hcsuf hc -static -H32m -O
-Werror -Wall -H64m -O0-package-name random-1.0.1.1 -hide-all-packages -i
-ilibraries/random/. -ilibraries/random/dist-install/build
-ilibraries/random/dist-install/build/autogen
-Ilibraries/random/dist-install/build
-Ilibraries/random/dist-install/build/autogen -Ilibraries/random/.
-optP-include -optPlibraries/random/dist-install/build/autogen/cabal_macros.h
-package base-4.7.0.0 -package time-1.4.2 -O2 -XHaskell98 -XCPP -O2 -O
-dcore-lint -fno-warn-amp -fno-warn-deprecated-flags -dcore-lint
-no-user-package-db -rtsopts -odir libraries/random/dist-install/build
-hidir libraries/random/dist-install/build -stubdir
libraries/random/dist-install/build -c libraries/random/./System/Random.hs -o
libraries/random/dist-install/build/System/Random.o
bash: [6536: 1 (255)] tcsetattr: Inappropriate ioctl for device
io
So this ioctl thing seems to be reported as a seg fault by the make system.
Running the same command from a 'cmd' window does give a seg-fault pop-up
window. Alas.
Running with -dshow-passes shows that it happens during code generation,
*** CorePrep:
Result size of CorePrep = {terms: 1,214, types: 912, coercions: 52}
*** Core Linted result of CorePrep:
*** Stg2Stg:
*** CodeOutput:
*** New CodeGen:
*** CPSZ:
*** CPSZ:
I am totally stuck.
Simon
From: Simon Peyton Jones
Sent: 06 March 2014 22:11
To: ghc-devs@haskell.org
Subject: new Windows build problem
Sigh. Totally new problem with building GHC (HEAD) on Windows..
inplace/bin/ghc-stage2.exe -hisuf hi -osuf o -hcsuf hc -static -H32m -O
-Werror -Wall -H64m -O0-package-name haskell98-2.0.0.3 -hide-all-packages
-i -ilibraries/haskell98/. -ilibraries/haskell98/dist-install/build
-ilibraries/haskell98/dist-install/build/autogen
-Ilibraries/haskell98/dist-install/build
-Ilibraries/haskell98/dist-install/build/autogen -Ilibraries/haskell98/.
-optP-include
-optPlibraries/haskell98/dist-install/build/autogen/cabal_macros.h -package
array-0.5.0.0 -package base-4.7.0.0 -package directory-1.2.0.2 -package
old-locale-1.0.0.6 -package old-time-1.1.0.2 -package process-1.2.0.0 -package
time-1.4.2 -Wall -XHaskell98 -O2 -O -dcore-lint -fno-warn-amp
-fno-warn-deprecated-flags -dcore-lint -no-user-package-db -rtsopts -odir
libraries/haskell98/dist-install/build -hidir
libraries/haskell98/dist-install/build -stubdir
libraries/haskell98/dist-install/build -c libraries/haskell98/./Directory.hs
-o libraries/haskell98/dist-install/build/Directory.o
bash: [6536: 1 (255)] tcsetattr: Inappropriate ioctl for device
HEAD $
What on earth is this ioctl stuff?
The above is repeatable; same command line, same result.
But I'm also getting random seg-faults when building libraries with the stage2
compiler
libraries/random/ghc.mk:5: recipe for target
'libraries/random/dist-install/build/System/Random.o' failed
make[1]: *** [libraries/random/dist-install/build/System/Random.o] Segmentation
fault
This is NOT repeatable. Saying 'make' again gets further.
Any ideas?
Simon
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Emitting calls to functions defined in PrimOps.cmm from the code gen
Hi, I'm trying to make allocation of arrays of small, statically-known size inline. For large or unknown size I just want to call the existing stg_newArrayzh function. In code: doNewArrayOp :: DynFlags - CmmFormal - CmmExpr - FCode () doNewArrayOp dflags res_r (CmmLit (CmmInt n _)) | n = inlineAllocLimit = do -- Do everything inline. doNewArrayOp dflags res_r n = do emitCallTo stg_newArrayzh res_r n -- HERE The question is: how do I emit a call to stg_newArrayzh, which is defined in PrimOps.cmm? -- Johan ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
