[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Termininja added a comment. Ups, sorry, I wanted to write 1 second, yes :) TASK DETAIL https://phabricator.wikimedia.org/T135471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: jcrespo, Termininja Cc: XXN, Termininja, Lydia_Pintscher,

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. Termininja, you do not need to write only once per minute, something like 1 POST/write per second is unlikely to cause issues, just wait for one edit to finish before starting the next one (DO NOT SENT MULTIPLE REQUEST IN PARALLEL). I will tell the admins on

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Termininja added a comment. Of course, this will solve the problems, in the next days I'll refactor my bot code. I think to make it to do all POST requests in serial, but to continue to use parallel for the GET requests. In any case in future will there is minimum 1 min interval between my

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. Termininja, let's try to fix this. Can you run edits exclusively in a serial way, as suggested, even if that means they will be slower? TASK DETAIL https://phabricator.wikimedia.org/T135471 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Legoktm added a comment. In https://phabricator.wikimedia.org/T135471#2305276, @Termininja wrote: > And yes Legoktm, I've never said that my bot use synchronous requests, why you have to believe in this? https://www.mediawiki.org/wiki/API:Etiquette#Request_limit says to make your

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Termininja added a comment. Thanks Aklapper, I'll use it if decide to do something in Commons. And yes Legoktm, I've never said that my bot use synchronous requests, why you have to believe in this? TASK DETAIL https://phabricator.wikimedia.org/T135471 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Legoktm added a comment. @Termininja: are you making edits in parallel or in series? I find it very hard to believe you'd be able to reach those speeds without using parallel requests. TASK DETAIL https://phabricator.wikimedia.org/T135471 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Aklapper added a comment. In https://phabricator.wikimedia.org/T135471#2300956, @Termininja wrote: > I need to know how much and what is criteria. Just tell me some **normal ** limit for edits per second (1, 2, 3 or ...) or critical median lag (10, 30, 50, ...) or maybe both.

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Termininja added a comment. The blocking is not problem, this is normal and right action when there is some problem with the bot, but don't forget that this bot is commanded by me, and I don't want to harm to any wiki, so keeping block on is of course unnecessarily and very stupid decision.

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. Independently of the policy, which says: > "There is no hard and fast limit on read requests, but we ask that you be considerate and try not to take a site down. Most sysadmins reserve the right to unceremoniously block you if you do endanger the stability of

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

Termininja added a comment. One year ago there was similar issue , then my bot made ~200 edits/sec but I didn't know anything about bot policies and the lags. When I was warned my bot started to checks median lag on every

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. A visual comparison: F4022609: ninja.png TASK DETAIL https://phabricator.wikimedia.org/T135471 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: jcrespo Cc: Lydia_Pintscher,

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. This bot was doing 10 edits per second, causing lag, specially on recentchanges on both wikidata and dewiki, and probably 10 in parallel. API policy does not set a concrete limit, but it is very clear about not being so

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. I think I found it: | COMMIT /* DatabaseBase::deadlockLoop BotNinja */ |0.000 | | NULL |0.000 |

[Wikidata-bugs] [Maniphest] [Commented On] T135471: Hundreds of concurrent SELECT MASTER_POS_WAIT('db1049-bin.002927', 354219766, 10) without debugging information (no user, not function) on s5 API no

jcrespo added a comment. Based on other errors, it could be batches of CategoryMembershipUpdates but just based on time frame (no conclusive evidence). It could be they just are affected at the same time. TASK DETAIL https://phabricator.wikimedia.org/T135471 EMAIL PREFERENCES