Lucas_Werkmeister_WMDE added a comment.
> Although batch size is 10, more than 10 things could actually end up being in a single batch in theory. I’m not sure why that batch size is there (it’s already present in the first patch set of the change introducing the class <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseQualityConstraints/+/349894/1/includes/UpdateConstraintsTableJob.php>). Perhaps we can just remove it? Most properties have less than 10 constraint statements, and none have more than 50 (summary <https://query.wikidata.org/#SELECT%20%3FconstraintStatementCount%20%28COUNT%28%3Fproperty%29%20AS%20%3FpropertyCount%29%20WITH%20%7B%0A%20%20SELECT%20%3Fproperty%20%28COUNT%28%3FconstraintStatement%29%20AS%20%3FconstraintStatementCount%29%20WHERE%20%7B%0A%20%20%20%20%3Fproperty%20a%20wikibase%3AProperty%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20p%3AP2302%20%3FconstraintStatement.%0A%20%20%7D%0A%20%20GROUP%20BY%20%3Fproperty%0A%7D%20AS%20%25properties%20WHERE%20%7B%0A%20%20INCLUDE%20%25properties.%0A%7D%0AGROUP%20BY%20%3FconstraintStatementCount%0AORDER%20BY%20ASC%28%3FconstraintStatementCount%29>), so your suggested size of 200 would effectively disable this level of batching anyways. I think it makes more sense to batch at the maintenance script level than in the job. > The most important thing here is probably to wait for replication lag somewhere which it doesn't look like the script or the job does. > It probably makes sense to do this just after the inserts, eg https://github.com/wikimedia/mediawiki-extensions-WikibaseQualityConstraints/blob/26846fd7320993f13570392232b6e358a3378a4f/src/UpdateConstraintsTableJob.php#L132 Does it make sense to wait for replication in a job, especially when most of the time the wait won’t be followed by another insert? I feel like that would just pointlessly block the job runner. (If the job runner wants to wait for replication before proceeding with the next job, that should be its own responsibility.) TASK DETAIL https://phabricator.wikimedia.org/T226635 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc: Addshore, Marostegui, Lucas_Werkmeister_WMDE, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Agabi10, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
