Igorkim78 added a comment.
Performance measured on dump from 20191202: https://dumps.wikimedia.org/wikidatawiki/entities/20191202/ Baseline tIme to load: 4264m29.914s, 714218864640 bytes Improvements proposed: 1. One-path loading (when data is loaded into SPO index only and POS, OSP are recreated in parallel afterwards). One-path time to load: 1755m57.082s (41.2% of baseline), 402815582208 bytes (56.4% of baseline) Indices recreation: In progress. 2. Data to be loaded is parsed in parallel, creating StatementBuffer instances, which then are queued for load into DB. To be done. TASK DETAIL https://phabricator.wikimedia.org/T236663 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Igorkim78 Cc: Addshore, Aklapper, Igorkim78, Gehel, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
