| Addshore added a comment. |
In T210044#4767023, @Smalyshev wrote:Maybe the Wikidata replica Updater is reading from is lagging behind the updates and returns an old revision? Not sure if that is possible. If it's possible to read old data after we've got Kafka message with new data, that would be a lot of trouble for us.
That seems totally possible to me, it's a shame the maxlag can not be requested by the updater while using Special:EntityData.
Most of the time the replicas are not lagged, but if your requesting the entity data sub 1 second hitting old data could be possible.
I went and looked at the requests of the updater.
"uri_path":"/wiki/Special:EntityData/Q60305784.ttl","uri_query":"?nocache=1550568436299&flavor=dump"
If I'm correct nocache here is just the current ts of the request added to avoid cache hits?
Although this will avoid hitting the varnish cache, you could still be getting old data in some situations.It might make sense to remove this and instead pass in either the oldid or revision param (if this data is available in kafka).
You'll then get a 404 if the revision is not yet retrievable and should probably wait a few seconds and try again, the correct amount of time to wait could probably be determined by maxlag, but maxlag doesn't currently work with Special:EntityData at all.
Cc: Addshore, MichaelSchoenitzer, doctaxon, Nikki, Lydia_Pintscher, hoo, Multichill, KaMan, Fnielsen, abian, Lea_Lacroix_WMDE, MisterSynergy, ValterVB, gerritbot, Graemebp, Gstupp, Tarrow, Jane023, EBjune, Floatingpurr, Wikidata-Query-Service, Oravrattas, Lucas_Werkmeister_WMDE, Stashbot, Alexsdutton, Aklapper, Smalyshev, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
