Hi, Am Freitag, den 09.12.2016, 13:54 +0800 schrieb Moritz Angermann: > > I am not sure what you are saying. Are you proposing the maintain a > > benchmark set outside GHC, or did you get the impression that I am > > proposing it? > > Yes, that’s what *I* am proposing for the reasons I mentioned; one I > did not yet mention is time. Running nofib takes time, adding more time > consuming performance tests would reduce their likelihood of being run > in my experience. As I see this as being almost completely scriptable, > this could live outside of ghc i think.
I don’t think the running time of nofib is a constraint at the moment, and I expect most who run nofib to happily let it run for a few minutes more in order to get more meaningful results. > > > But maybe it is ok if it part of nofib, and hence of GHC, so that every > > breaking change in GHC can immediately be accounted for in the > > benchmark code. > > > > A nice side effect of this might be that GHC developers can get a > > better idea of how much code their change breaks. > > I’m not much a fan of this, but that’s just my opinion :-) What is the alternative? Keep updating the libraries? But libraries change APIs. Then you need to keep updating the program itself? That seems to be too many moving parts for a benchmark suite. > > > Something I’ve recently had some success with was dumping measurements > > > into influxdb[1] (or a similar data point collections service) and hook > > > that up to grafana[2] for visualization. > > > > Nice! Although these seem to be tailored for data-over-time, not > > data-over-commit. This mismatch in the data model was part of the > > motivation for me to create gipeda, which powers > > https://perf.haskell.org/ghc/ > > Assuming we confine this to a particular branch, or discriminate by branch, > commits would be measured in sequence anyway, and the timestamp could be the > time of the reporting of the measurement, and the respective ghc commit hash > end up being an annotation. While this is not very pretty (and I would hope > that grafana has some other ability to enrich the hover-tooltips) it could > present a flexible solution without requiring additional engineering effort. > > However, if gipeda is sufficient, please ignore my comment :) Oh, we could certainly do better here! (But it serves my purposes for now, so I’ll stick to it until someone sets up something better, in which case I happily dump my code.) Greetings, Joachim -- Joachim “nomeata” Breitner m...@joachim-breitner.de • https://www.joachim-breitner.de/ XMPP: nome...@joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F Debian Developer: nome...@debian.org
signature.asc
Description: This is a digitally signed message part
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs