[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #20 from Helder --- (In reply to Nemo from comment #19) > If you're willing to merge it, that's ok. But let's see how > I1da936c786adb21e2c1802ef405bb904c9cf4918 goes. That patch was merged. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #19 from Nemo --- Thanks Reedy for the cleanup. Tim said: > Or just set watchdefault=1 by default for all users. If you're willing to merge it, that's ok. But let's see how I1da936c786adb21e2c1802ef405bb904c9cf4918 goes. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Kunal Mehta (Legoktm) changed: What|Removed |Added CC||legoktm.wikipe...@gmail.com --- Comment #18 from Kunal Mehta (Legoktm) --- (In reply to Tim Starling from comment #17) > (In reply to Nemo from comment #14) > > Tim, it's still not clear to me what alternative you propose for bug 36316 > > not to be a problem. (We've been blocked one more year just to save some > > thousands DB rows per day?) > > Have a default that depends on user registration date. We should be able to do this easily for extensions with the new hook added in I1da936c786adb21e2c1802ef405bb904c9cf4918. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #17 from Tim Starling --- (In reply to Nemo from comment #14) > Tim, it's still not clear to me what alternative you propose for bug 36316 > not to be a problem. (We've been blocked one more year just to save some > thousands DB rows per day?) Have a default that depends on user registration date. Or just set watchdefault=1 by default for all users. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bug 52777 depends on bug 64756, which changed state. Bug 64756 Summary: Remove nocache from user_properties with runBatchedQuery.php on all wikis https://bugzilla.wikimedia.org/show_bug.cgi?id=64756 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bug 52777 depends on bug 64755, which changed state. Bug 64755 Summary: Remove disablesuggest from user_properties with runBatchedQuery.php on all wikis https://bugzilla.wikimedia.org/show_bug.cgi?id=64755 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bug 52777 depends on bug 64754, which changed state. Bug 64754 Summary: Remove ajaxsearch from user_properties with runBatchedQuery.php on all wikis https://bugzilla.wikimedia.org/show_bug.cgi?id=64754 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bug 52777 depends on bug 64753, which changed state. Bug 64753 Summary: Remove showjumplinks from user_properties with runBatchedQuery.php on all wikis https://bugzilla.wikimedia.org/show_bug.cgi?id=64753 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bug 52777 depends on bug 64752, which changed state. Bug 64752 Summary: Remove vector-collapsiblenav from user_properties with runBatchedQuery.php on all wikis https://bugzilla.wikimedia.org/show_bug.cgi?id=64752 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bug 52777 depends on bug 64757, which changed state. Bug 64757 Summary: Remove searchlimit from user_properties with runBatchedQuery.php on all wikis https://bugzilla.wikimedia.org/show_bug.cgi?id=64757 What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||52542 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #16 from Nemo --- > (In reply to Nemo from comment #14) > > [...] please file one shell request for each. > > I will not do that. The list is not closed, so maybe there are more > preferences to be removed. Hence I said one for each. :) Anyway, easier done than said. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||64757 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||64755 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||64756 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||64754 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||64753 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Depends on||64752 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #15 from Umherirrender --- (In reply to Nemo from comment #14) > How? Umherirrender, if there is a script already suitable for that[...] See comment #2: (In reply to Tim Starling from comment #2) > You can't really delete 9M rows in one query, it will cause replication lag. > But the generic maintenance script runBatchedQuery.php can be used. (In reply to Nemo from comment #14) > [...] please file one shell request for each. I will not do that. The list is not closed, so maybe there are more preferences to be removed. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added CC||sprin...@wikimedia.org --- Comment #14 from Nemo --- Tim, it's still not clear to me what alternative you propose for bug 36316 not to be a problem. (We've been blocked one more year just to save some thousands DB rows per day?) > You can remove the 'vector-collapsiblenav' preference from database, because > it was removed > ajaxsearch doesn't seem to be used anymore either, there's another 769131 > rows that can be dropped. > You can remove 'disablesuggest' preference from the database, because it was > removed from core > You can remove 'searchlimit' preference from the database, was removed from > core How? Umherirrender, if there is a script already suitable for that, please file one shell request for each. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #13 from Umherirrender --- You can remove 'searchlimit' preference from the database, was removed from core with gerrit 105871. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #12 from Umherirrender --- Expand a existing maintenance script to clean up the database: gerrit 101233 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #11 from Max Semenik --- https://gerrit.wikimedia.org/r/100938 should improve the situation a bit. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #10 from Umherirrender --- You can remove 'nocache' preference from the database, was removed from core with gerrit 98276. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #9 from Umherirrender --- You can remove 'disablesuggest' preference from the database, because it was removed from core with gerrit 99163 You may find more by looking at the fixed bugs which depends on bug 52807 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Danny B. changed: What|Removed |Added CC||dann...@email.cz --- Comment #8 from Danny B. --- Not only wrong values, but also wrong keys are stored (need to find my data, adding this as a reminder). Also - deleted or renamed gadgets are still stored as well (I am not sure, but I think I submitted it as a bug, but I clearly remember also being told by one of the devs that "it's not a bug, it's a feature"). -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #7 from Chad H. --- ajaxsearch doesn't seem to be used anymore either, there's another 769131 rows that can be dropped. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #6 from Umherirrender --- You can remove the 'vector-collapsiblenav' preference from database, because it was removed from vector extension and not migrated to core: gerrit 83591 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Bartosz Dziewoński changed: What|Removed |Added Blocks||36316 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Umherirrender changed: What|Removed |Added CC||umherirrender_de...@web.de --- Comment #5 from Umherirrender --- You can remove the showjumplinks preference from the database, because it was removed from core: gerrit 25751 Due to the watchlist token stored in the user_properties table for each user which has visited the watchlist exists one row. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 db changed: What|Removed |Added CC||duplicate...@googlemail.com --- Comment #4 from db --- There is also a userOptions.php which will delete rows, when you change a preferences to a default state, but you have to run it for each old value seperate. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Nemo changed: What|Removed |Added Blocks||29782 --- Comment #3 from Nemo --- (In reply to comment #0) > We should remove unnecessary or incorrectly inserted rows, and ensure that > this > does not happen again (e.g. as a consequence of the resolution of bug 36316). How to avoid that it happens again? Just changing a default should not have this effect, right? What about changing a default only for new users via post-registration hook as done for some things now? Thumb default was changed in 2010 with bug 21117. On en.wiki there were also some load tests before the actual, global switch. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 Andre Klapper changed: What|Removed |Added Keywords||performance Priority|Unprioritized |Normal -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #2 from Tim Starling --- (In reply to comment #1) > I'm not sure correcting these database table row anomalies will require a > maintenance script. The cleanup for "searchNs-1" and "thumbsize" can probably > be done with a simple query across all.dblist. However, if a maintenance > script is needed for either of these, separate bugs should be filed as > dependencies of this bug. You can't really delete 9M rows in one query, it will cause replication lag. But the generic maintenance script runBatchedQuery.php can be used. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 --- Comment #1 from MZMcBride --- Thank you for researching this. (In reply to comment #0) > Sampling 1000 users with user_id<10M, we find that the main culprits are: > > searchNs-1 : 968 users > skin: 964 users > thumbsize: 912 users > > 75% of the skin rows have an empty string as their value, which causes > Skin::newFromKey() to return the default skin, same as if the row was > missing. The rest are mostly "monobook", presumably manually set via the > UsabilityInitiative OptIn extension. I've filed bug 52778 to track the skin issue, as I think it's more complex than the other two issues mentioned and I think it should be resolved with a maintenance script to allow third-party MediaWiki users to clean up their own databases. > "searchNs-1" is a bug, it relates to searching the special namespace, which > is not possible. It is "0" in all sampled rows. > > "thumbsize" is "3" in all sampled rows, which is not the default, the default > is "4" on all WMF wikis other than svwiki. In addition to bloat of the > user_properties table, this causes fragmentation of the parser cache. There's > no way 91% of users prior to 2009 manually set this value, it must have been > set by a bug. I'm not sure correcting these database table row anomalies will require a maintenance script. The cleanup for "searchNs-1" and "thumbsize" can probably be done with a simple query across all.dblist. However, if a maintenance script is needed for either of these, separate bugs should be filed as dependencies of this bug. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 52777] user_properties table bloat
https://bugzilla.wikimedia.org/show_bug.cgi?id=52777 MZMcBride changed: What|Removed |Added Depends on||52778 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
