| jcrespo added a comment. |
I think the materialization is a wrong approach, and we should try to materialize the changes only on query. E.g.:
Watchlist x page x wikidata_usage JOINS being queried on real time. Of course, that makes the query an order of magnitude slower and more complex but- some tricks could be used, and memory is probably going to be faster than pure on-disk materialization. That would be the general idea, I do not think it is easy, but at least it would not affect queries not requiring including Wikidata changes and it would not be dangerous for other reasons (see NDA related tickets).
Recentchanges has already been affected because bad performance queries where possible in the past, with the new materialized entries, those are not taking over 60 seconds to execute and fail to return to the user (you can see the rate of connection lost on API queries to recentchanges).
Cc: Mattflaschen-WMF, Strainu, TerraCodes, Jay8g, Liuxinyu970226, jcrespo, Bawolff, Aklapper, hoo, daniel, Ladsgroup, Lydia_Pintscher, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
