[Wikidata-bugs] [Maniphest] [Commented On] T115222: Compress JSON data dumps in Bzip2

2015-10-13 Thread gerritbot
gerritbot added a subscriber: gerritbot. gerritbot added a comment. Change 245850 had a related patch set uploaded (by Hoo man): Publish bzip2 compressed Wikidata json dumps https://gerrit.wikimedia.org/r/245850 TASK DETAIL https://phabricator.wikimedia.org/T115222 EMAIL PREFERENCES https:

[Wikidata-bugs] [Maniphest] [Commented On] T115222: Compress JSON data dumps in Bzip2

2015-10-11 Thread Halfak
Halfak added a comment. xz does not have the nice built in support in distributed processing frameworks that bz2 has. It may be worth re-iterating that I am not concerned about compression ratio. The purpose of this task is to make wikidata JSON dumps easy to process in Hadoop/Spark. A quick