I'm glad you brought this up, Oisín. I was already thinking of
appealing to this mailing list, in hopes of finding an eager detective
to hunt down what is going on! I can say that I can achieve much better
performance with the latest code on my own workstation (similar profile
to /one/ of the several machines used by TechEmpower), which leads me to
think something basic is getting in the way of proper performance in the
benchmarking environment.
On 7/31/19 8:06 PM, Oisín Mac Fhearaí wrote:
I've noticed that Ur/web's performance benchmarks on Techempower have
changed significantly between round 16 and 17.
For example, in round 16, Urweb measured 323,430 responses per second
to the "Fortunes" benchmark.
In round 17 (and beyond), it achieved 4,024 RPS with MySQL and 2,544
RPS with Postgres.
What could explain such a drastic drop in performance? The blog entry
for round 17 mentioned query pipelining as an explanation for some of
the frameworks getting much faster, but I don't see why Urweb's RPS
would drop by a factor of 100x, unless perhaps previous rounds had
query caching enabled and then round 17 disabled them.
Can anyone here shed light on this? I built a simplified version of
the "sql" demo with the 2016 tarball version of Ur (used by the round
16 benchmarks) and a recent snapshot, and they both perform at similar
speeds on my laptop.
Oddly, the load testing tool I used (a Go program called "hey") seems
to have one request that takes 5 seconds if I set it to use more
concurrent threads than the number of threads available to the Ur/web
program. Otherwise, the longest request takes about 0.02 seconds. This
seems unrelated to the performance drop on the Techempower benchmarks,
since the max latency is quite low there.
_______________________________________________
Ur mailing list
Ur@impredicative.com
http://www.impredicative.com/cgi-bin/mailman/listinfo/ur
