[Wikidata-bugs] [Maniphest] [Updated] T181385: Wikidata entity dumpers stuck with 100% CPU on snapshot1007

[hoo]

hoo added a comment.

So I finally managed to track this down:

In Revision::getRevisionText, MainWANObjectCache::getWithSetCallback is used to retrieve revisions.

As of 4432e898be078eea73fbf072f5a383ec1ded0d67 WANObjectCache uses a StatsdDataFactoryInterface for certain kinds of (often very high volume) logging. Given MediaWiki uses a BufferingStatsdDataFactory per default, we basically try to accumulate all these stat hits until we eventually run out of memory.

Just commenting out $this->buffer[] = $entity; in BufferingStatsdDataFactory::produceStatsdData (locally on snapshot1001) immediately fixed this.

TASK DETAIL

https://phabricator.wikimedia.org/T181385

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo
Cc: TerraCodes, Jay8g, Liuxinyu970226, Stashbot, Aklapper, daniel, Lydia_Pintscher, ArielGlenn, aude, hoo, Lahi, Gq86, GoranSMilovanovic, QZanden, Wikidata-bugs, Svick, Mbch331, jeremyb

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs