Re: Testing GHC against Hackage via CI
Richard Eisenberg writes: > How will this affect workflow of developers submitting patches? For > example, suppose I write a user-facing change that breaks some of > these packages. Am I expected to patch up the breakage? How? Will the > CI infrastructure detect this before merging or after? > > To be clear, I don't have specific answers I'm looking for here. In > particular, I think it's reasonable to ask that the developer of a > user-facing feature make sure that head.hackage works with the > feature. This has the distinct advantage of making sure the developer > knows that their patch breaks code and how it must be fixed. Indeed, > this can all be fodder to force the developer to update the GHC > migration guide alongside their work in fixing head.hackage. I just > want to know what the steps are. :) > All very good questions. Frankly, many of them are still open. I should have made this clearer: The short answer is that in the short term very little is changing. head.hackage is, at the moment, merely another tool that we can use to evaluate contributions and identify regressions in the compiler; it's not (yet) a requirement that MRs pass the job. The reason is that I don't think we are yet in a position to start requiring contributors to update head.hackage. For one, the workflow for doing so is arguably not well enough documented (something that I will be working to fix). Moreover, it's quite unclear to me exactly how much work maintaining this infrastructure is going to require. I'd like to better understand this before requiring that contributors put in the work. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Testing GHC against Hackage via CI
How will this affect workflow of developers submitting patches? For example, suppose I write a user-facing change that breaks some of these packages. Am I expected to patch up the breakage? How? Will the CI infrastructure detect this before merging or after? To be clear, I don't have specific answers I'm looking for here. In particular, I think it's reasonable to ask that the developer of a user-facing feature make sure that head.hackage works with the feature. This has the distinct advantage of making sure the developer knows that their patch breaks code and how it must be fixed. Indeed, this can all be fodder to force the developer to update the GHC migration guide alongside their work in fixing head.hackage. I just want to know what the steps are. :) Thanks, Richard PS: And thanks, generally, to all of you doing the heavy lifting to get this infrastructure in place. This is all quite wonderful, even if there have been some bumps in the road en route! > On Mar 5, 2019, at 1:10 PM, Ben Gamari wrote: > > Hello everyone, > > As many of you know, recently I have been working on bringing up > infrastructure for using CI-produced binary distributions to build a > subset of Hackage using the head.hackage patchset. > > Happily, this effort has now converged on a usable result, embodied in > two merge requests: > > * https://gitlab.haskell.org/ghc/head.hackage/merge_requests/2 > * https://gitlab.haskell.org/ghc/ghc/merge_requests/465 > > ghc/head.hackage!2 adds CI support to head.hackage. In addition to the > usual pass/fail status, this job produces (e.g. [1]) a few additional > products: > > * a JSON summary of the run, describing the dependency graph and > pass/fail state of each package. we can feed to an external service to > track newly-failing packages. > > * a (rendered) graphviz graph showing the dependency structure of the > built packages, each node colored by its pass/fail state > > * a tarball of build logs, each including statistics from > -ddump-timings. The hope here is that we can use this to track > compiler performance on real-world code > > ghc/ghc!465 adds a job to GHC's own CI pipeline to trigger a > head.hackage job using the binary distribution produced earlier in the > pipeline. This job will run automatically under a variety of > circumstances: > > * via the scheduled nightly pipeline > > * on merge requests labelled with the `user-facing` label > > * when manually started using a button on the MR's Pipelines tab > > Currently the head.hackage MR tests only a small number of Hackage > packages (and their dependencies). Specifically, we currently build > `aeson`, `criterion`, `singletons`, `servant`, and `scotty`. This pulls > in 127 packages in total. > > There are a few things that remain to be done: > > * Better document the existence of the job and how it is triggered > > * Document how to update the list of tested packages > > * Work out how to handle tracking of persistent breakage (e.g. we want > a notification when a package initially breaks but not in later > builds) > > * Automatically push patched packages to a Hackage repository (e.g. > head.hackage.org) as a result of CI. > > Cheers, > > - Ben > > > [1] https://gitlab.haskell.org/ghc/head.hackage/-/jobs/38022 > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Testing GHC against Hackage via CI
Ryan Scott writes: > This is fantastic work! I'm looking forward to using this. > > Are there plans to merge GitLab's fork of head.hackage back into the > upstream repo [1]? I ask since I regularly submit patches to upstream, but > if the GitLab CI is using the fork, then it may take some time for upstream > changes to make their way back to the fork. > Yes, if it's okay with Herbert I think this branch should be merged upstream. We might also consider going even further: making gitlab.haskell.org/ghc/head.hackage the upstream repository. If we don't do this I would likely need to maintain a "stable" branch alongside `master` to ensure that the head.hackage build doesn't break due to bad patches landing in `master`. Moving the upstream to GitLab would avoid this since all patches landed would go through CI. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Testing GHC against Hackage via CI
This is fantastic work! I'm looking forward to using this. Are there plans to merge GitLab's fork of head.hackage back into the upstream repo [1]? I ask since I regularly submit patches to upstream, but if the GitLab CI is using the fork, then it may take some time for upstream changes to make their way back to the fork. Ryan S. - [1] https://github.com/hvr/head.hackage ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Testing GHC against Hackage via CI
Hello everyone, As many of you know, recently I have been working on bringing up infrastructure for using CI-produced binary distributions to build a subset of Hackage using the head.hackage patchset. Happily, this effort has now converged on a usable result, embodied in two merge requests: * https://gitlab.haskell.org/ghc/head.hackage/merge_requests/2 * https://gitlab.haskell.org/ghc/ghc/merge_requests/465 ghc/head.hackage!2 adds CI support to head.hackage. In addition to the usual pass/fail status, this job produces (e.g. [1]) a few additional products: * a JSON summary of the run, describing the dependency graph and pass/fail state of each package. we can feed to an external service to track newly-failing packages. * a (rendered) graphviz graph showing the dependency structure of the built packages, each node colored by its pass/fail state * a tarball of build logs, each including statistics from -ddump-timings. The hope here is that we can use this to track compiler performance on real-world code ghc/ghc!465 adds a job to GHC's own CI pipeline to trigger a head.hackage job using the binary distribution produced earlier in the pipeline. This job will run automatically under a variety of circumstances: * via the scheduled nightly pipeline * on merge requests labelled with the `user-facing` label * when manually started using a button on the MR's Pipelines tab Currently the head.hackage MR tests only a small number of Hackage packages (and their dependencies). Specifically, we currently build `aeson`, `criterion`, `singletons`, `servant`, and `scotty`. This pulls in 127 packages in total. There are a few things that remain to be done: * Better document the existence of the job and how it is triggered * Document how to update the list of tested packages * Work out how to handle tracking of persistent breakage (e.g. we want a notification when a package initially breaks but not in later builds) * Automatically push patched packages to a Hackage repository (e.g. head.hackage.org) as a result of CI. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/head.hackage/-/jobs/38022 signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
