Lucas_Werkmeister_WMDE created this task. Lucas_Werkmeister_WMDE added projects: Wikidata, Wikibase-Quality-Constraints. Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION As a deployer, I want to run the `extensions/WikibaseQualityConstraints/maintenance/ImportConstraintStatements.php` maintenance script when necessary in order to resolve problems with the constraint definitions, without having to worry about its impact on the wiki. **Problem:** The maintenance script currently imports constraint statements for all properties in a simple loop, with no offset, limit, batching, or sleeping. This only takes a few minutes – less than 10 ms for most properties, not much more for some larger properties, and we’re still well below 10000 properties – but it was enough to cause a visible traffic spike on the s8 master db host <https://grafana.wikimedia.org/d/000000273/mysql?panelId=5&fullscreen&orgId=1&var-dc=eqiad%20prometheus%2Fops&var-server=db1071&var-port=9104&from=1561554000000&to=1561555200000> during the deployment for T223372 <https://phabricator.wikimedia.org/T223372> (around 13:17 UTC). As Wikidata keeps getting more properties, I believe this will gradually become more important. **Example:** We could group the imports into batches (with some default batch size, e. g. 100) and sleep between each batch (default e. g. 3 s). Batch size and sleep time should be configurable via command line arguments. Some offset/limit arguments could also be added, so that a deployer can first import constraint statements for the first 1000 properties, then proceed later with the next 1000, and so on. **BDD** //can probably be removed, but keeping the template in case someone wants to fill it in// GIVEN AND WHEN AND THEN AND **Acceptance criteria:** - The rate of action of the maintenance script is configurable - The default does not put unreasonable strain on the database **Open questions:** - Is batch+sleep the best approach? Do we need offset+limit? - What should the default batch size and sleep time be? TASK DETAIL https://phabricator.wikimedia.org/T226635 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc: Lucas_Werkmeister_WMDE, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Agabi10, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
