On 2016-12-08 17:04, Joachim Breitner wrote: > Hi, > > Am Donnerstag, den 08.12.2016, 01:03 -0500 schrieb Joachim Breitner: >> I am not sure how useful this is going to be: >> + Tests lots of common and important real-world libraries. >> − Takes a lot of time to compile, includes CPP macros and C code. >> (More details in the README linked above). > > another problem with the approach of taking modern real-world code: > It uses a lot of non-boot libraries that are quite compiler-close and > do low-level stuff (e.g. using Template Haskell, or stuff like the). If > we add that not nofib, we’d have to maintain its compatibility with GHC > as we continue developing GHC, probably using lots of CPP. This was > less an issue with the Haskell98 code in nofib. > > But is there a way to test realistic modern code without running into > this problem? >
This may be a totally crazy idea, but has any thought been given a "Phone Home"-type model? Very simplistic approach: a) Before it compiles, GHC computes a hash of the file. b) GHC has internal profiling "markers" in its compilation pipeline. c) GHC sends those "markers" + hash to some semi-centralized highly-available service somewhere under *.haskell.org. The idea is that the fact that "hashes are equal" => "performance should be comparable". Ideally, it'd probably be best to be able to have the full source, but that may be a tougher sell, obviously. (Obviously would have to be opt-in, either way.) There are a few obvious problems with this, but an obvious win would be that it could be done on a massively *decentralized* scale. Most problematic part might be that it wouldn't be able to track things like "I changed $this_line and now it compiles twice as slow". Actually, now that I think about it: What about if this were integrated into the Cabal infrastructure? If I specify "upload-perf-numbers: True" in my .cabal file, any project on (e.g.) GitHub that wanted to opt-in could do so, they could build using Travis, and voila! What do you think? Totally crazy, or could it be workable? Regards, _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs