[Wikidata-bugs] [Maniphest] [Commented On] T176844: Look into truthy nt dump performance

2017-09-29 Thread hoo
hoo added a comment. With PHP 5.5.9-1ubuntu4.22 on snapshot1005: 6 alternating runs with --sharding-factor 5 --shard 0 --limit 2000 (ttl vs. nt): (62.110+65.767+63.446+63.018+66.504+64.367)/(43.654+44.974+47.964+44.570+47.304+48.088) = 1.393 Optimized nt vs. ttl: (56.080+60.318+58.797+55.146+60.4

[Wikidata-bugs] [Maniphest] [Commented On] T176844: Look into truthy nt dump performance

2017-09-29 Thread hoo
hoo added a comment. Using sql wikidatawiki "SELECT page_title FROM (SELECT page_title, page_namespace FROM page ORDER BY page_id DESC LIMIT 110) AS 1Mpages WHERE page_namespace = 0 ORDER BY rand() LIMIT 2500" I got a random list of 2500 recently created Items. I just re-did the above tests wi

[Wikidata-bugs] [Maniphest] [Commented On] T176844: Look into truthy nt dump performance

2017-09-29 Thread hoo
hoo added a comment. I just benchmarked this some more: I did 6 alternating runs with nt and ttl with --sharding-factor 8 --shard 0 --limit 1000 on mwdebug1001: (38.077+33.798+33.044+34.293+33.142+34.518)/(23.859+24.070+23.178+23.632+23.666+23.450) = 1.458 After that I applied the optimizations

[Wikidata-bugs] [Maniphest] [Commented On] T176844: Look into truthy nt dump performance

2017-09-29 Thread gerritbot
gerritbot added a comment. Change 381229 merged by jenkins-bot: [purtle@master] Skip simple strings in N3Quoter::escapeLiteral https://gerrit.wikimedia.org/r/381229TASK DETAILhttps://phabricator.wikimedia.org/T176844EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T176844: Look into truthy nt dump performance

2017-09-28 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2017-09-28T15:22:48Z] Ran scap pull on mwdebug1001 after experiments with T176844TASK DETAILhttps://phabricator.wikimedia.org/T176844EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc

[Wikidata-bugs] [Maniphest] [Commented On] T176844: Look into truthy nt dump performance

2017-09-28 Thread gerritbot
gerritbot added a comment. Change 381229 had a related patch set uploaded (by Hoo man; owner: Hoo man): [purtle@master] Skip simple strings in N3Quoter::escapeLiteral https://gerrit.wikimedia.org/r/381229TASK DETAILhttps://phabricator.wikimedia.org/T176844EMAIL PREFERENCEShttps://phabricator.wikim