Attempt at a real world benchmark

[Joachim Breitner]

Hi,

I have talked so much about it, it was about time to actually follow
through.

I took a real-world program (tttool, one of mine, show shameless),
inlined all the dependencies so that it would compile without Cabal,
just with a single invocation of GHC on the 277 modules.

The state, including a README, can be found at 
https://github.com/nomeata/tttool-nofib#turning-tttool-into-a-benchmark

I am not sure how useful this is going to be:
 + Tests lots of common and important real-world libraries.
 − Takes a lot of time to compile, includes CPP macros and C code.
(More details in the README linked above).

It is late, so I’ll just send this out as it is for now to get the
discussion going if this is a useful approach, or what should be done
different.

(If this is deemed to be useful, I’d do the same for, say, pandoc
next).

Greetings,
Joachim

-- 
Joachim “nomeata” Breitner
  m...@joachim-breitner.de • https://www.joachim-breitner.de/
  XMPP: nome...@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F
  Debian Developer: nome...@debian.org

signature.asc
Description: This is a digitally signed message part
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: How to inline early in a GHC plugin?

[Conal Elliott]

Yes, thank you, Simon. I had not occurred to me that an inlining could
start working in a later phase due to loss of loop-breaker status, rather
than the relationship between the current phase and the identifier's
declared inlining phase.

-- Conal

On Tue, Dec 6, 2016 at 6:02 AM, Simon Peyton Jones 
wrote:

> *   To access an unfolding, is `maybeUnfoldingTemplate (idUnfolding v)`
> the recommended recipe?
>
> You can see by looking at the code that idUnfolding returns nothing for a
> loop breaker.  You have to decide if that’s what you want; if not, use
> realIdUnfolding.
>
>
>
> *   Is it the case that this recipe succeeds (`Just`) in some compiler
> phases and not others?
>
> It fails for loop breakers.  An Id might be a loop breaker in some phases
> but not others; e.g. the loop might be broken by some optimisation.
>
>
>
> *   Before an Id is ready for general inlining by the simplifier, can I
> get the Id's unfolding another way so that I can substitute it early?
>
>
>
> realIdUnfolding always works.  As the code shows
>
>
>
> idUnfolding :: Id -> Unfolding
>
> -- Do not expose the unfolding of a loop breaker!
>
> idUnfolding id
>
>   | isStrongLoopBreaker (occInfo info) = NoUnfolding
>
>   | otherwise  = unfoldingInfo info
>
>   where
>
> info = idInfo id
>
>
>
> realIdUnfolding :: Id -> Unfolding
>
> -- Expose the unfolding if there is one, including for loop breakers
>
> realIdUnfolding id = unfoldingInfo (idInfo id)
>
>
>
>
>
> Does that help?
>
>
>
> Simon
>
>
>
>
>
> *From:* conal.elli...@gmail.com [mailto:conal.elli...@gmail.com] *On
> Behalf Of *Conal Elliott
> *Sent:* 02 December 2016 18:13
> *To:* Simon Peyton Jones 
> *Cc:* ghc-devs@haskell.org
> *Subject:* Re: How to inline early in a GHC plugin?
>
>
>
> Thanks for the pointers, Simon. Some more specific questions:
>
>
>
> *   To access an unfolding, is `maybeUnfoldingTemplate (idUnfolding v)`
> the recommended recipe?
>
> *   Is it the case that this recipe succeeds (`Just`) in some compiler
> phases and not others?
>
> If so, is this difference due to Ids being altered (presumably via
> `setUnfoldingInfo` being called between phases)?
>
> *   Before an Id is ready for general inlining by the simplifier, can I
> get the Id's unfolding another way so that I can substitute it early?
>
>
>
> A short Skype chat might easily clear up my questions and confusions if
> you have time and inclination.
>
>
>
> Regards, - Conal
>
>
>
> On Fri, Dec 2, 2016 at 9:07 AM, Simon Peyton Jones 
> wrote:
>
> I don’t really understand your question clearly.  So I’ll guess
>
>
>
> Unfoldings are added to Ids in Simplify.completeBind (look for
> setUnfoldingInfo).  Apart from INLINE pragmas, that’s about the only place
> it happens.
>
>
>
> Does that help?
>
>
>
> S
>
>
>
> *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org] *On Behalf Of *Conal
> Elliott
> *Sent:* 01 December 2016 21:51
> *To:* ghc-devs@haskell.org
> *Subject:* How to inline early in a GHC plugin?
>
>
>
> I'm implementing a GHC plugin that installs a `BuiltInRule` that does the
> work, and I'd like to learn how to inline more flexibly. Given an
> identifier `v`, I'm using `maybeUnfoldingTemplate (realIdUnfolding v)` to
> get a `Maybe CoreExpr`. Sometimes this recipe yields `Nothing` until a
> later compiler phase. Meanwhile, I guess my variable `v` has been replaced
> by one with inlining info. First, am I understanding this mechanism
> correctly? A GHC source pointer to how inlining is made available would
> help me. Second, can I access the inlining info before it's made available
> to the rest of the simplifier?
>
>
>
> Thanks,  - Conal
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Michal Terepeta]

On Tue, Dec 6, 2016 at 10:10 PM Ben Gamari  wrote:
> [...]
> > How should we proceed? Should I open a new ticket focused on this?
> > (maybe we could try to figure out all the details there?)
> >
> That sounds good to me.

Cool, opened: https://ghc.haskell.org/trac/ghc/ticket/12941 to track
this.

Cheers,
Michal
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Ben Gamari]

Johannes Waldmann  writes:

> Hi Ben, thanks,
>
>
>>  4. run the build, `cabal configure --ghc-options="-p -hc" $args && cabal 
>> build`
>
> cabal configure $args --ghc-options="+RTS -p -hc -RTS"
>
Ahh, yes, of course. I should have tried this before hitting send.

>> You should end up with a .prof and .hp file.
>
> Yes, that works. - Typical output starts like this
>
> COST CENTRE   MODULE   %time %alloc
>
> SimplTopBinds SimplCore 60.7   57.3
> OccAnal   SimplCore  6.06.0
> Simplify  SimplCore  3.00.5
>
Ahh yes. So one of the things I neglected to mention is that the
profiled build flavour includes only a few cost centers. One of the
tricky aspects of the cost-center profiler is that it affects
core-to-core optimizations, meaning that the act of profiling may
actually shift around costs. Consequently by default the 
the build flavour includes a rather conservative set of cost-centers to
avoid distorting the results and preserve compiler performance.

Typically when I've profiled the compiler I already have a region of
interest in mind. I simply add `OPTIONS_GHC -fprof-auto` pragmas to the
modules involved. The build system already adds this flag to a few
top-level modules, hence the cost-centers which you observe (see
compiler/ghc.mk; search for GhcProfiled).

If you don't have a particular piece of the compiler in mind to study,
you certainly can just pepper every module with cost centers by adding
-fprof-auto to GhcStage2HcOpts (e.g. in mk/build.mk). The resulting
compiler may be a bit slow and you may need to be just a tad more
careful in evaluating the profile.

It might be nice if we had a more aggressive profiled build flavour
which added cost centers to a larger fraction of machinery of the
compiler, which excluding low-level utilities like FastString, which are
critical to the compiler's performance.

>
> These files are always called ghc.{prof,hp},
> how could this be changed? Ideally, the output file name
> would depend on the package being compiled,
> then the mechanism could probably be used with 'stack' builds.
>
We really should have a way to do this but sadly do not currently.
Ideally we would also have a way to change the default eventlog
destination path.

> Building executables mentioned in the cabal file will
> already overwrite profiling info from building libraries.
>
Note that you can instruct `cabal` to only build a single component of a
package. For instance, in the case of the `text` package you can build
just the library component with `cabal build text`.

> When I 'cabal build' the 'text' package,
> then the last actual compilation (which leaves
> the profiling info) is for cbits/cbits.c
>
Ahh right. Moreover, there is likely another GHC invocation after that
to link the final library. This is why I typically just use GHC
directly, perhaps stealing the command line produced by `cabal` (with
`-v`).

> I don't see how to build Data/Text.hs alone
> (with ghc, not via cabal), I am getting
> Failed to load interface for ‘Data.Text.Show’
>
Hmm, I'm not sure I see the issue. In the case of `text` I can just run
`ghc` from the source root (ensuring that I set the #include path with
`-I`),

$ git clone git://github.com/bos/text
$ cd text
$ ghc Data/Text.hs -Iinclude


However, some other packages (particularly those that make heavy use of
CPP) aren't entirely straightforward. In these cases I often find myself
copying bits from the command line produced by cabal.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Joachim Breitner]

Hi,

Am Mittwoch, den 07.12.2016, 11:34 +0100 schrieb Johannes Waldmann:
> When I 'cabal build' the 'text' package,
> then the last actual compilation (which leaves
> the profiling info) is for cbits/cbits.c
> 
> I don't see how to build Data/Text.hs alone
> (with ghc, not via cabal), I am getting
> Failed to load interface for ‘Data.Text.Show’

you can run
$ cabal build -v
and then copy’n’paste the command line that you are intested in, add
the flags
+RTS -p -hc -RTS -fforce-recomp
and run that again.

Greetings,
Joachim


-- 
Joachim “nomeata” Breitner
  m...@joachim-breitner.de • https://www.joachim-breitner.de/
  XMPP: nome...@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F
  Debian Developer: nome...@debian.org

signature.asc
Description: This is a digitally signed message part
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Johannes Waldmann]

Hi Ben, thanks,


>  4. run the build, `cabal configure --ghc-options="-p -hc" $args && cabal 
> build`

cabal configure $args --ghc-options="+RTS -p -hc -RTS"


> You should end up with a .prof and .hp file.

Yes, that works. - Typical output starts like this

COST CENTRE   MODULE   %time %alloc

SimplTopBinds SimplCore 60.7   57.3
OccAnal   SimplCore  6.06.0
Simplify  SimplCore  3.00.5


These files are always called ghc.{prof,hp},
how could this be changed? Ideally, the output file name
would depend on the package being compiled,
then the mechanism could probably be used with 'stack' builds.

Building executables mentioned in the cabal file will
already overwrite profiling info from building libraries.

When I 'cabal build' the 'text' package,
then the last actual compilation (which leaves
the profiling info) is for cbits/cbits.c

I don't see how to build Data/Text.hs alone
(with ghc, not via cabal), I am getting
Failed to load interface for ‘Data.Text.Show’


- J.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs