Thanks Todd, that makes more sense now. I gave up on trying to build the native libraries on os x (not officially supported presumably because it's such a PITA) and instead ran from a centos machine and that worked flawlessly out of the box.
Cheers, Oliver On 2012-01-09, at 7:21 PM, Todd Lipcon wrote: > On Mon, Jan 9, 2012 at 2:42 AM, Oliver Meyn (GBIF) <om...@gbif.org> wrote: >> It seems really weird that compression (native compression even moreso) >> should be required by a command that is in theory moving files from one >> place on a remote filesystem to another. Any light shed would be >> appreciated. > > The issue is that the completebulkload script does actually open the > files to read their metadata as well as the first/last key in the > file. This is necessary to figure out which region each file belongs > in. So, you do need the compression support on whatever machine you > run completebulkload from. > > -Todd > -- > Todd Lipcon > Software Engineer, Cloudera > -- Oliver Meyn Software Developer Global Biodiversity Information Facility (GBIF) +45 35 32 15 12 http://www.gbif.org
