dr0ptp4kt added a comment.
Here's the output from the latest run based upon a larger set of queries from
a random sample of WDQS queries.
$ /usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java -cp iguana-3.3.3.jar
org.aksw.iguana.rp.analysis.TabularTransform -e result.nt > result.execution.csv
$ cut -f1,3,5,6,7,9 -d"," result.execution.csv | sed 's/,/|/g'
| endpointLabel | taskStartDate | successfullQueries |
successfullQueriesQPH | avgqps | queryMixesPH |
| ------------------- | ------------------------ | ------------------ |
--------------------- | ------ | ------------ |
| baseline | 2024-01-31T23:20:44.567Z | 319857 |
136612.71246575614 | 18.83670491311007 | 1.732300885924224
|
| wikidata_main_graph | 2024-02-01T04:23:01.613Z | 331473 |
147674.12233239523 | 19.55930142298825 | 1.8725637484770261
|
|
Here's the screen capture from Grafana.
F41740308: Screenshot 2024-02-01 at 10.17.28 AM.png
<https://phabricator.wikimedia.org/F41740308>
The `wikidata_main_graph` window completed more queries despite an apparent
bout of increased failing queries (climb began at about 0915 UTC), with a large
garbage collection beginning about 5 minutes later (GC started at about 0920
UTC; the GC actually continued well after the `wikidata_main_graph`'s window
closure at 2024-02-01T09:23:55.639Z). This isn't the most interesting thing as
it only constitutes about 1.5%-3.0% of the `wikidata_main_graph` window
depending on how one looks at it, and I wouldn't necessarily read anything into
whether such GCs would be likely to occur under the same conditions, but I
wanted to note it nonetheless.
To repeat the verbiage from the earlier runs...
> Following below are "per-query" summary stats. I actually just put this
together by bringing CSV data into Google Sheets for now - all of the columns
are calculated upon the "per-query" rows (but you'll see how the Mean
corresponds basically with the value calculated up above). The underlying CSV
data don't bear actual queries (the .nt files from which they're generated do),
...
The CSV data were generated with the following command:
`/usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java -cp iguana-3.3.3.jar
org.aksw.iguana.rp.analysis.TabularTransform -q result.nt > result.query.csv`
| Run | Endpoint Label | Mean | Median | Standard Deviation |
Max (fastest) | 99% (very fast) | 0.95 | 0.75 | 0.5 | 0.25 | 1% (pretty slow)
| Total w/ success |
| ------------ | ------------------- | ---- | ------ | ------------------ |
------------- | --------------- | ---- | ---- | ---- | ---- | ----------------
| ---------------- |
| randomized 1 | baseline | 18.8367049131101 | 14.6999663404689
| 16.3589173757083 | 127.433177227691 | 59.009472115968
| 50.5734395961334 | 30.3470335487675 | 14.6999663404689 |
4.97164300568995 | 0 | 319857 |
| randomized 1 | wikidata_main_graph | 19.5593014229883 | 16.0982853987134
| 16.5098295290687 | 121.141149629509 | 58.9613256488317
| 51.0426872548935 | 31.751311031492 | 16.0982853987134 |
5.37249826361878 | 0 | 331473 |
|
Although the max and 99th percentile queries were just ever so slightly
faster on the baseline "full" graph, more generally things were faster on the
non-scholarly "main" graph. The performance difference is obvious but not
dramatic.
Here's the content of `wdqs-split-test-randomized-2024-01-31.yml`, comments
removed for brevity. The main difference in this configuration file from the
earlier presented one is five hours allowed per graph, to accommodate a larger
query mix, and the updated filename pointing to the larger query mix based on
the set of queries from the random sample.
datasets:
- name: "split"
connections:
- name: "baseline"
endpoint: "https://wdqs1022.eqiad.wmnet/sparql";
- name: "wikidata_main_graph"
endpoint: "https://wdqs1024.eqiad.wmnet/sparql";
tasks:
- className: "org.aksw.iguana.cc.tasks.impl.Stresstest"
configuration:
timeLimit: 18000000
warmup:
timeLimit: 30000
workers:
- threads: 4
className: "SPARQLWorker"
queriesFile:
"queries_for_performance_file_renamed_randomized_2024_01_31.txt"
timeOut: 5000
queryHandler:
className: "DelimInstancesQueryHandler"
configuration:
delim: "### BENCH DELIMITER ###"
workers:
- threads: 4
className: "SPARQLWorker"
queriesFile:
"queries_for_performance_file_renamed_randomized_2024_01_31.txt"
timeOut: 60000
parameterName: "query"
gaussianLatency: 100
metrics:
- className: "QMPH"
- className: "QPS"
- className: "NoQPH"
- className: "AvgQPS"
- className: "NoQ"
storages:
- className: "NTFileStorage"
configuration:
fileName: result.nt
TASK DETAIL
https://phabricator.wikimedia.org/T355037
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dr0ptp4kt
Cc: dr0ptp4kt, dcausse, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1,
karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi,
Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen,
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
