On 16/12/2021 10:08, Marco Neumann wrote:
thank you Lorenz, I am running this test myself now again with a larger
disk. You may want to consider running a full load of wikidata as well. The
timing info and disk space you have should be sufficient.
Full Wikidata (WD).
I've tried to gather a summary as of 2021-12.
The video [2] is most up-to-date at the time of writing.
WD about 16B currently.
It's growing at 1B/3 months [1]
The query service is 6 active machines (11 in total)
58 queries per server per second average (no figures for peak)
12 wikidata updates/330 triples per second (no figures for peak)
It seems this mixed workload is causing the most pain.
Wikidata will want at least a 5 year strategy, and planning for 10 years
isn't unreasonable. The service is 6 year old.
That's 36B and 56B as a baseline *minimum*.
That's without new streams of data.
They can't design for a minimum. "plan for success"
--> towards 100B triples.
Andy
[1]
https://wikitech.wikimedia.org/wiki/Wikidata_Query_Service/ScalingStrategy
[2] Scaling the Wikidata Query Service
https://www.youtube.com/watch?v=oV4qelj9fxM
