Hey everyone! I'm not a frequent contributor but I observed similar challenges as Hécate. I notice couple points.
### HLS and other editor integrations I've never tried HLS for GHC development but it absolutely chokes on Cabal for me (a $2K laptop), so I'm not surprised it's having troubles with GHC. I've never tried to dig into it, but I heard before that disabling plugins is a good start. Ghcid (after the introduction of ghc-in-ghci) was interesting but still on a slow side. I once tried to generate ETAGS and use them from Emacs (with plain haskell-mode): this was quite nice. As Moritz, I didn't use much above syntax coloring, but ETAGS allowed jumping to definitions, which is important. Maintaining tags wasn't fun, on the other hand. In all fairness, I think that's an issue with HLS more than with GHC. ### Build Times I have been using a dedicated server for this, but this still was painful at times (even git clone takes non-negligible amount of time, and I never got used to git worktree because of a hoop you have to jump over, which I already forgot but I know it can be looked up in Andreas Herrmann's presentation on developing GHC). I'm surprised no one seems to try to challenge the status quo. Hadrian is a Shake application. How is Cloud Shake doing? In the era of Nix and Bazel you start assuming niceties like remote caching. It'd be great to improve on this front as it just feels very wrong rebuilding master again and again on every contributor's computer. Especially after so much effort put into GHC modularity, which, I believe, should make it easier to cache. It's sad that GHC still needs ./boot && ./configure: this can preclude any remote caching technology that I can imagine. At one point it seemed like configure could go into Hadrian, but it didn't really happen. In the bright future I see Hadrian doing the whole job and using remote cache on GitLab CI. Not sure how big of a task that would be. -- Cheers, Artem On Tue, 19 Jul 2022 at 15:11, Ben Gamari <[email protected]> wrote: > Hécate <[email protected]> writes: > > > Hello ghc-devs, > > > > I hadn't made significant contributions to the GHC code base in a while, > > until a few days ago, where I discovered that my computer wasn't able to > > sustain running the test suite, nor handle HLS well. > > > > Whether it is my OS automatically killing the process due to oom-killer > > or just the fact that I don't have a war machine, I find it too bad and > > I'm frankly discouraged. > > Do you know which process was being killed? There is one testsuite tests > that I know of which does have quite a considerable memory footprint > (T16992) due to its nature; otherwise I would expect a reasonably recent > machine to pass the testsuite without much trouble. It's particularly > concerning if this is a new regression; is this the first time you have > observed this particular failure? > > > This is not the first time such feedback emerges, as the documentation > > task force for the base library was unable to properly onboard some > > people from third-world countries who do not have access to hardware > > we'd consider "standard" in western Europe or some parts of North > > America. Or at least "standard" until even my standard stuff didn't cut > > it anymore. > > > > So yeah, I'll stay around but I'm afraid I'm going to have to focus on > > projects for which the feedback loop is not on the scale of hours , as > > this is a hobby project. > > > > Hope this will open some eyes. > > > Hi Hécate, > > I would reiterate that the more specific feedback you can offer, the > better. > > To share my some of my own experience: I have access to a variety of > hardware, > some of which is quite powerful. However, I find that I end up doing > much of my development on my laptop which, while certainly not a slouch > (being a Ryzen 4750U), is also not a monster. In particular, while a > fresh build takes nearly twice as long on my laptop than some of the > other hardware I have, I nevertheless find ways to make it worthwhile > (due to the ease of iteration compared to ssh). If you routinely have > multi-hour iteration times then something isn't right. > > In particular, I think there are a few tricks which make life far > easier: > > > * Be careful about doing things that would incur > significant amounts of rebuilding. This includes: > > * After modifying, e.g., `compiler/ghc.cabal.in` (e.g. to add a new > module to GHC), modify `compiler/ghc.cabal` manually instead of > rerunning `configure`. > > * Be careful about pulling/rebase. I generally pick a base commit to > build off of and rebase sparingly: Having to stop what I'm doing to > wait for full rebuild is an easy way to lose momentum. > > * Avoid switching branches; I generally have a GHC tree per on-going > project. > > * Take advantage of Hadrian's `--freeze1` flag > > * Use `hadrian/ghci` to typecheck changes > > * Use the stage1 compiler instead of stage2 to smoke-test changes when > possible. (specifically, using the script generated by Hadrian's > `_build/ghc-stage1` target) > > * Use the right build flavour for the task at hand: If I don't need a > performant compiler and am confident that I can get by without > thorough testsuite validation, I use `quick`. Otherwise, plan ahead > for what you need (e.g. `default+assertions+debug_info` or > `validate`) > > * Run the fraction of the testsuite that is relevant to your change. > Hadrian's `--test-way` and `--only` flags are your friends. > > * Take advantage of CI. At the moment we have a fair amount of CI > capacity. If you think that your change is close to working, you can > open an MR and start a build locally. If it fails, iterate on just the > failing testcases locally. > > * Task-level parallelism. Admittedly, this is harder when you are > working as a hobby, but I often have two or three projects on-going > at a time. While one tree is building I try to make progress on > another. > > I don't use HLS so I may be insulated from some of the pain in this > regard. However, I do know that Matt is a regular user and he > disables most plugins. > > I would also say that, sadly, GHC is comparable to other similarly-size > compilers in its build time: A build of LLVM (not even clang) takes ~50 > minutes on my 8-core desktop; impressively, rustc takes ~7 minutes > although it is a considerably smaller compiler (being just a front-end). > By contrast, GHC takes around 20 minutes. I know that this doesn't > make the cost any easier to bear and I would love to bring this number > down, but ultimately there are only so many hours in the day. > > I think one underexplored approach to addressing the build-time problem > is to look not at the full-build time but rather look for common tasks > where we could *avoid* doing a full build (e.g. updating documentation, > typechecking `base`, running a "good enough" subset of the testsuite) > and find ways to make those workflows more efficient. > > Cheers, > > - Ben > > _______________________________________________ > ghc-devs mailing list > [email protected] > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >
_______________________________________________ ghc-devs mailing list [email protected] http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
