[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-11-10 Thread gerritbot
gerritbot added a comment.
Change 383107 abandoned by Reedy:
Disable inject recent changes on all client wikis

https://gerrit.wikimedia.org/r/383107TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Bawolff, gerritbotCc: JEumerus, Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Lahi, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-17 Thread Vicpeters
Vicpeters added a comment.
Thanks for help! It works now!TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Bawolff, VicpetersCc: JEumerus, Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-13 Thread Jwh
Jwh added a comment.
Works fine now.
Thank you so much to have solved this problem!TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: JwhCc: JEumerus, Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-13 Thread jcrespo
jcrespo added a comment.
The initial scope- query issues on ruwiki and commonswiki, I think has been successfully patched. These are the statistics I have now:

F10184141: Screenshot_20171013_165442.png

I would ask users affected to confirm the issues are gone- probably not 100% of them will disappear forever, but I am confident most will- to levels of 3 months ago (not better, not worse). On separate tickets we can handle the followups of how to reenable the hosts, checking other potential projects affected and reclaiming disk space.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespoCc: JEumerus, Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-12 Thread Johan
Johan added a comment.
Will fix.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, JohanCc: Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-12 Thread Catrope
Catrope added a comment.

In T171027#3680606, @IKhitron wrote:
Hi, @Johan. Recent changes or watchlists?


Both. But the statement should really make that clearer. It uses RC and WL as if they're interchangeable, and in the code they kind of are because they use the same underlying infrastructure, but most non-technical people won't know that.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, CatropeCc: Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-12 Thread Johan
Johan added a comment.
@jcrespo Is this an accurate description right now?

There is a problem with watchlists that show a large number of pages. Until it has been fixed Wikidata edits will not be shown in recent changes on Commons or Russian Wikipedia. If necessary Wikidata edits will be removed from watchlists on more wikis.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, JohanCc: Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-12 Thread Catrope
Catrope added a comment.

In T171027#3677449, @Catrope wrote:
Thanks, I'll experiment with removing the STRAIGHT_JOIN, as well as with @awight's suggestion at T164796#3560530 of breaking out the change tags rollup.


This is tracked over at T176456: ORES on Watchlist causes big slowdown—especially with 'Last revision' filter turned on, but just to close the loop here: removing the STRAIGHT_JOIN (on watchlist but not RC) appears to work. I wrote a patch for that, it was merged, and I'm SWATting it to wmf.3 in the upcoming SWAT window in 15 minutes.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, CatropeCc: Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-12 Thread jcrespo
jcrespo added a comment.
This is the result on query error rate after ruwiki has been purged:

F10158519: Screenshot_20171012_130744.png

Please note that these are preliminary results, and that tables have yet to be optimized/analyzed, but I think this confirms we are in the right track.

Commons has both more rows to purge and more load, so it will take more time to finish the purging (still ongoing).TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespoCc: Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-11 Thread Alsee
Alsee added a comment.

In T171027#3673060, @Lydia_Pintscher wrote:
This is a hugely political issue.


It doesn't matter whether Political Issues come from the community, a project manager, or President of the Planet. They wait in line when a database administrator declares an emergency:
we are close of running out of space on several main database servers, breaking all of Wikipedia.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, AlseeCc: Alsee, awight, Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-11 Thread Bawolff
Bawolff added a comment.
Well wikidata is almost certainly a contributing factor, (particularly for russian) I would hesitate to blame it solely for slow ores on watchlist speeds, without more evidence. Especially on english wikipedia with its large recentchanges table and relatively low usage of wikidata.

So addendum to this. For the enwiki case, and talking about watchlist (Not Special:RecentChanges) all the timeouts I could find - https://logstash.wikimedia.org/goto/3183d6a077082f6b4469255e51e60105 (This are just the severe read timeouts. I had trouble making a logstash query for the query longer than 5 seconds warning) had STRAIGHT_JOIN specified, including people with small watchlists. This probably makes sense for Special:Recentchanges case, but for the watchlist case, when the user has a small watchlist, this will cause the query to be much slower then needed. I'd reccomend removing the STRAIGHT_JOIN in the watchlist case for ORESTASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, BawolffCc: Matthewrbowker, Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-11 Thread Bawolff
Bawolff added a comment.

In T171027#3676023, @jmatazzoni wrote:
In T171027#3673060, @Lydia_Pintscher wrote:

This is a hugely political issue. Let's please not do this unless necessary.

Right now, ORES is unusable on Watchlist for English, Russian and probably some other big wikis. It causes delays of 30 seconds to a minute when users have more than 3 days set. I'm told Wikidata is a likely culprit. If that's correct (I can't say it is or isn't), then by that standard, some kind of change appears to be "necessary,"  and, as Risker says, some kind of escalation appears both justified and required.


Well wikidata is almost certainly a contributing factor, (particularly for russian) I would hesitate to blame it solely for slow ores on watchlist speeds, without more evidence. Especially on english wikipedia with its large recentchanges table and relatively low usage of wikidata.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, BawolffCc: Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-11 Thread jmatazzoni
jmatazzoni added a comment.
In T171027#3673060, @Lydia_Pintscher wrote:

This is a hugely political issue. Let's please not do this unless necessary.

Right now, ORES is unusable on Watchlist for English, Russian and probably some other big wikis. It causes delays of 30 seconds to a minute when users have more than 3 days set. I'm told Wikidata is a likely culprit. If that's correct, then by that standard, some kind of change appears to be "necessary,"  and, as Risker says, some kind of escalation appears both justified and required.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, jmatazzoniCc: Noella94, Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-10 Thread Risker
Risker added a comment.

In T171027#3673060, @Lydia_Pintscher wrote:
This is a hugely political issue. Let's please not do this unless necessary. Daniel has provided a patch for T177707 and Marius has been working on various other improvements.


I'm not sure how political it is at this point; it was an issue, certainly, when use of Wikidata was first being considered on some projects, and a regression to remove it from recent changes and/or watchlists may have negative impacts on a few projects.  On the other hand, removing it entirely, with noise coming from communities about it, will in fact focus the minds to address this issue - not only for Wikidata, but also for ORES and for Commons changes as well.  Two are in place now and are having an impact, and the third has been a much-desired feature for as long as I can remember.  Getting this right is important and worthy of the investment of time of many knowledgeable developers, not just one or two people trying to do a quick "repair" without digging into the underlying issues.  Including all of these things in some sort of easily accessible recent changes/watchlist profile is very important, particularly for projects with a larger community; however, the way we are doing things now (particularly on projects where the recent changes origins are heavily weighted toward Wikidata) is creating major bottlenecks.  The methodology for including this data deserves some rethinking and testing of other options.  We can't just keep tacking more tables onto watchlists/recent changes, if the impact is rendering RC/watchlists slow to the point of unusable or flooding the servers.

It strikes me that this is an unforeseen effect of the remarkable success of Wikidata. If not for its extensive usage on certain projects, the root causes of this bug would not have been identified so quickly.  There is every reason to believe that if other projects continue to expand their use of Wikidata, they too will encounter the same issues; the same is true if a similar methodology is used to include Commons changes.  This problem isn't really about Wikidata; it's about the effects of the ever-expanding use of recent changes.  It's just particularly obvious with Wikidata because it has been so successful.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, RiskerCc: Nirmos, Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis

2017-10-10 Thread Lydia_Pintscher
Lydia_Pintscher added a comment.
This is a hugely political issue. Let's please not do this unless necessary. Daniel has provided a patch for T177707 and Marius has been working on various other improvements.TASK DETAILhttps://phabricator.wikimedia.org/T171027EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespo, Lydia_PintscherCc: Stryn, Mike_Peel, Capankajsmilyo, Elitre, Risker, Peachey88, Stashbot, Finavon, WMDE-leszek, saper, Masti, Lydia_Pintscher, D3r1ck01, matej_suchanek, Ankry, Ladsgroup, Lsanabria, Josve05a, Bawolff, greg, Dominicbm, Vriullop, Jmabel, Fae, IKhitron, Johan, Herzi.Pinki, jmatazzoni, Mattflaschen-WMF, KTC, Framawiki, zhuyifei1999, Marostegui, aaron, Andrei_Stroe, Turbojet, Rsocol, Strainu, Jwh, Pyb, Darwinius, Arbnos, Jdforrester-WMF, TheDJ, gerritbot, VladXe, Kf8, Liuxinyu970226, Jay8g, TerraCodes, Iniquity, jcrespo, Reedy, Catrope, Vicpeters, Demidenko, MaxBioHazard, Aklapper, Waytogoeducation, Lordiis, GoranSMilovanovic, Adik2382, Abiyoyo, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Vali.matei, Jack_who_built_the_house, Lewizho99, Minhnv-2809, Maathavan, Poyekhali, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, Base, aude, GWicke, El_Grafo, Gryllida, putnik, Steinsplitter, Mbch331, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs