[Wikidata-bugs] [Maniphest] [Updated] T177486: [Tracking] Wikidata entity dumpers need to cope with the immense Wikidata growth recently

[hoo]

hoo added a comment.

Run time for the full TTL dump (the data diff is from the gzipped file):

20170918: 25:30h (5 shards)
20170925: 26:30h (5 shards) (~ +3.7% data)
20171009: 30:30h (5 shards) (~ +6.7% data)
20171016: 36:30h (6 shards) (~ +1.9% data) (contains ad332804b1fea069043d14d0195f6fe2ed5a6f4b and 3164215d0d790f37cc1cf386ef22a188e81a10d0)
20171023: 43:10h (6 shards) (~ +0.8% data)
20171030: 26:00h (6 shards) (~ +1.6% data)

This also indicates that our current changes had a huge impact… but also that ad332804b1fea069043d14d0195f6fe2ed5a6f4b and/ or 3164215d0d790f37cc1cf386ef22a188e81a10d0 might have a huge negative impact here :/

TASK DETAIL

https://phabricator.wikimedia.org/T177486

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo
Cc: Sjoerddebruin, gerritbot, thiemowmde, Aklapper, ezachte, daniel, Lydia_Pintscher, mark, ArielGlenn, bd808, Liuxinyu970226, aude, JanZerebecki, Jimkont, Denis.bykov, Ricordisamoa, PokestarFan, hoo, Lahi, GoranSMilovanovic, QZanden, Wikidata-bugs, Svick, Mbch331, jeremyb

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs