Zbyszko added a comment.
Few details on the issue: - For extensively modified entities (e.g. by bots) log show that information provided by RecentChanges API isn't always up to date. This can lead to lost revisions - if the last change was among the ones not yet provided by the API, the next check will omit it, since it thinks that previous time period was already handled. - Our workaround for the issue is to add the delay to processing time periods of changes. This isn't an ideal solution for two reasons - it adds a lag to an already substantial process and it doesn't provide guarantees that the delay length we set will help in all cases. - We won't be pursuing a better solution for this issue with current updater - we're closing in on having a new, streaming based, updater on production. It uses Kafka events and Flink to reconcile the updates which allows for a much less fragile process. - This doesn't fix the currently inconsistent entities, but that will be done with the data reload - T267927 <https://phabricator.wikimedia.org/T267927>. TASK DETAIL https://phabricator.wikimedia.org/T267175 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko Cc: Epidosis, dcausse, Tagishsimon, Lydia_Pintscher, CBogen, Z_thomas, agray, Gehel, Lucas_Werkmeister_WMDE, Aklapper, M2k_dewiki, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
