Simon Peyton Jones via ghc-devs <[email protected]> writes:

> When I run `sh validate -legacy`, towards the end I see stuff like this
>
> Performance Metrics (test environment: local):
>
>
>
> Conversions(normal)       runtime/bytes allocated                        
> 107696.000
>
> DeriveNull(normal)        runtime/bytes allocated                     
> 112050960.000
>
> InlineArrayAlloc(normal)  runtime/bytes allocated                    
> 1600041088.000
>
> InlineByteArrayAlloc(normal)  runtime/bytes allocated                    
> 1440041088.000
>
> InlineCloneArrayAlloc(normal)  runtime/bytes allocated                    
> 1600041248.000
>
> ManyAlternatives(normal)  compile_time/bytes allocated                
> 840985728.000
>
> ManyConstructors(normal)  compile_time/bytes allocated               
> 4540766560.000
>
> MethSharing(normal)       runtime/peak_megabytes_allocated                    
> 2.000
>
> MethSharing(normal)       runtime/bytes allocated                     
> 480098136.000
>
> MultiLayerModules(normal)  compile_time/bytes allocated               
> 5856970504.000
>
> ...
> It is intermingled with other apparently unrelated output.

Hmm, it really should be a distinct block of output. Are you saying that
you are seeing lines of unrelated output interspersed in the performance
metrics table?

> What should I conclude form this? Is it good or bad? By what amount
> have these figures changed, and relative to what?

Below the performance metrics table you should see a blurb of text like
the following:


    Missing Baseline Metrics these metrics trivially pass because a
    baseline (expected value) cannot be recovered from previous git
    commits. This may be due to HEAD having new tests or having expected
    changes, the presence of expected changes since the last run of the
    tests, and/or the latest test run being too old.

          MultiLayerModules
          ...

    If the tests exist on the previous commit (And are configured to run
    with the same ways), then check out that commit and run the tests to
    generate the missing metrics. Alternatively, a baseline may be
    recovered from ci results once fetched:

          git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git 
refs/notes/perf:refs/notes/ci/perf

The suggested command will fetch up-to-date performance metrics from the
metrics repository (populated by CI). If you then run the testsuite
again you will see output comparing each test's output to the baseline
from CI. For instance,

    Performance Metrics (test environment: local):

    MultiLayerModules(normal)  compile_time/bytes allocated               
5710260920.000
                               (baseline @ HEAD~1)                        
5726848340.000  [unchanged 0.3%]

The "unchanged" here refers to the fact that the +0.3% observed change
is within the indicated acceptance window of the test.


> How can I run a single perf test?

Perf tests are treated like any other test. A single test can be run
under Hadrian with the following:

    ./hadrian/build.cabal.sh test --build-root=_validatebuild \
      --flavour=Validate --only=MultiLayerModules

Cheers,

- Ben

Attachment: signature.asc
Description: PGP signature

_______________________________________________
ghc-devs mailing list
[email protected]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Reply via email to