Thanks Moritz for that update.

The latest is that currently darwin CI is disabled and the merge train
is unblocked (*choo choo*).

I am testing Moritz's patches to speed-up CI and will merge them in
shortly to get darwin coverage back.

Cheers,

Matt

On Wed, May 19, 2021 at 9:46 AM Moritz Angermann
<moritz.angerm...@gmail.com> wrote:
>
> Matt has access to the M1 builder in my closet now. The darwin performance 
> issue
> is mainly there since BigSur, and (afaik) primarily due to the amount of 
> DYLD_LIBRARY_PATH's
> we pass to GHC invocations. The system linker spends the majority of the time 
> in the
> kernel stat'ing and getelements (or some similar directory) call for each and 
> every possible
> path.
>
> Switching to hadrian will cut down the time from ~5hs to ~2hs. At some point 
> we had make
> builds <90min by just killing all DYLD_LIBRARY_PATH logic we ever had, but 
> that broke
> bindists.
>
> The CI has time values attached and some summary at the end right now, which 
> highlights
> time spent in the system and in user mode. This is up to 80% sys, 20% user, 
> and went to
> something like 20% sys, 80% user after nuking all DYLD_LIBRARY_PATH's, with 
> hadrian it's
> closer to ~25% sys, 75% user.
>
> Of note, this is mostly due to time spent during the *test-suite*, not the 
> actual build. For the
> actual build make and hadrian are comparable, though I've seen hadrian to 
> oddly have a
> much higher variance in how long it takes to *build* ghc, whereas the make 
> build was more
> consistent.
>
> The test-suite quite notoriously calls GHC *a lot of times*, which makes any 
> linker issue due
> to DYLD_LIBRARY_PATH (and similar lookups) much worse.
>
> If we would finally split building and testing, we'd see this more clearly I 
> believe. Maybe this
> is motivation enough for someone to come forward to break build/test into two 
> CI steps?
>
> Cheers,
>  Moritz
>
> On Wed, May 19, 2021 at 4:14 PM Matthew Pickering 
> <matthewtpicker...@gmail.com> wrote:
>>
>> Hi all,
>>
>> The darwin pipelines are gumming up the merge pipeline as they are
>> taking over 4 hours to complete on average.
>>
>> I am going to disable them -
>> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5785
>>
>> Please can someone give me access to one of the M1 builders so I can
>> debug why the tests are taking so long. Once I have fixed the issue
>> then I will enable the pipelines.
>>
>> Cheers,
>>
>> Matt
_______________________________________________
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Reply via email to