I tried cloning the latest version of the benchmarks to run the Urweb tests locally, but sadly the Docker image fails to build for me (due to a problem with the Postgres installation steps, it seems). I've opened an issue here: https://github.com/TechEmpower/FrameworkBenchmarks/issues/4969 ... I also asked for advice on how to track down the massive performance drop in the Urweb tests. Hopefully they might have some thoughts on it. Sadly I'm running things on a 9 year old laptop so it's hard to draw conclusions around performance...
On Thu, 1 Aug 2019 at 13:23, Adam Chlipala <[email protected]> wrote: > I'm glad you brought this up, Oisín. I was already thinking of appealing > to this mailing list, in hopes of finding an eager detective to hunt down > what is going on! I can say that I can achieve much better performance > with the latest code on my own workstation (similar profile to *one* of > the several machines used by TechEmpower), which leads me to think > something basic is getting in the way of proper performance in the > benchmarking environment. > On 7/31/19 8:06 PM, Oisín Mac Fhearaí wrote: > > I've noticed that Ur/web's performance benchmarks on Techempower have > changed significantly between round 16 and 17. > > For example, in round 16, Urweb measured 323,430 responses per second to > the "Fortunes" benchmark. > In round 17 (and beyond), it achieved 4,024 RPS with MySQL and 2,544 RPS > with Postgres. > > What could explain such a drastic drop in performance? The blog entry for > round 17 mentioned query pipelining as an explanation for some of the > frameworks getting much faster, but I don't see why Urweb's RPS would drop > by a factor of 100x, unless perhaps previous rounds had query caching > enabled and then round 17 disabled them. > > Can anyone here shed light on this? I built a simplified version of the > "sql" demo with the 2016 tarball version of Ur (used by the round 16 > benchmarks) and a recent snapshot, and they both perform at similar speeds > on my laptop. > > Oddly, the load testing tool I used (a Go program called "hey") seems to > have one request that takes 5 seconds if I set it to use more concurrent > threads than the number of threads available to the Ur/web program. > Otherwise, the longest request takes about 0.02 seconds. This seems > unrelated to the performance drop on the Techempower benchmarks, since the > max latency is quite low there. > > _______________________________________________ > Ur mailing list > [email protected] > http://www.impredicative.com/cgi-bin/mailman/listinfo/ur >
_______________________________________________ Ur mailing list [email protected] http://www.impredicative.com/cgi-bin/mailman/listinfo/ur
