[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 574863 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@master] Do prefetching entity ids on batches of 9 entity per request https://gerrit.wikimedia.org/r/574863 TASK DETAIL https://phabricator.wikim

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Marostegui added a comment. In T246159#5917713 , @gerritbot wrote: > Change 574863 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): > [mediawiki/extensions/Wikibase@master] Do prefetching entity ids on batches of 9 e

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Ladsgroup added a comment. In T246159#5918782 , @Marostegui wrote: > In T246159#5917713 , @gerritbot wrote: > >> Change 574863 had a related patch set uploaded (by Ladsgroup; owner:

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

jcrespo added a comment. Do you have latency data for each run? Normally clients report query execution time, which you should already have when getting the handlers. TASK DETAIL https://phabricator.wikimedia.org/T246159 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Ladsgroup added a comment. In T246159#5919315 , @jcrespo wrote: > Do you have latency data for each run? Normally clients report query execution time, which you should already have when getting the handlers. - For the main thing: 25

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

jcrespo added a comment. I am almost made up my mind to give a suggestion, last question, I promise: could you provide an idea of frequency of this query (normally & at peak times). This will be the determinant factor on how much latency we can "spend" on it. I don't need exact values, just

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Ladsgroup added a comment. In T246159#5919507 , @jcrespo wrote: > I am almost made up my mind to give a suggestion, last question, I promise: could you provide an idea of frequency of this query (normally & at peak times). This will be

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

jcrespo added a comment. so 200K per minute overall is approximately 20/60/3 (db1126 has a third of the main, non special traffic) = 1000 QPS max per server. I would categorize this as frequent, normally those should have a latency of 0.01 seconds or less in average (and for that, that m

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Ladsgroup added a comment. Thanks, I put 20 for now. In T246159#5919892 , @jcrespo wrote: > The issue is that if that amount of reads have to happen anyway, just they will be sent in batches, unless we slow down the query rate someh

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 574863 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Do prefetching entity ids on batches of 20 entity per query https://gerrit.wikimedia.org/r/574863 TASK DETAIL https://phabricator.wikimedia.org/T246159 EMAIL PREFERENCES http

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 575054 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.21] Do prefetching entity ids on batches of 20 entity per query https://gerrit.wikimedia.org/r/575054 TASK DETAIL https://phabric

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 575055 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.20] Do prefetching entity ids on batches of 20 entity per query https://gerrit.wikimedia.org/r/575055 TASK DETAIL https://phabric

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 575054 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.21] Do prefetching entity ids on batches of 20 entity per query https://gerrit.wikimedia.org/r/575054 TASK DETAIL https://phabricator.wikimedia.org/T246159 EMAIL PREFERE

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 575055 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.20] Do prefetching entity ids on batches of 20 entity per query https://gerrit.wikimedia.org/r/575055 TASK DETAIL https://phabricator.wikimedia.org/T246159 EMAIL PREFERE

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-02-26T20:35:53Z] Synchronized php-1.35.0-wmf.21/extensions/Wikibase/lib/includes/Store/Sql/Terms: SWAT: [[gerrit:575055|Do prefetching entity ids on batches of 20 entity per query (T246159

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 575082 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [operations/mediawiki-config@master] Revert read from the new term store again to Q2M https://gerrit.wikimedia.org/r/575082 TASK DETAIL https://phabricator.wikimedia.org/T2461

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

gerritbot added a comment. Change 575082 **merged** by jenkins-bot: [operations/mediawiki-config@master] Revert read from the new term store again to Q2M https://gerrit.wikimedia.org/r/575082 TASK DETAIL https://phabricator.wikimedia.org/T246159 EMAIL PREFERENCES https://phabricat

[Wikidata-bugs] [Maniphest] [Commented On] T246159: Setting on read new for items causes increased worst response time and number of open connections

Ladsgroup added a comment. So fun fact, the given doesn't de-duplicate entities. This needs to be fixed, After de-duplication, it's 655 which is better. For the whole thing, the handler stats are: | Variable_name | Value | | Handler_commit | 1 | | Handle