reducers and data locality

Hello folks, I have a lot of input splits (10k-50k - 128 mb blocks) which contains text files. I need to process those line by line, then copy the result into roughly equal size of "shards". So i generate a random key (from a range of [0:numberOfShards]) which is used to route the map output to d

Re: reducers and data locality

: common-user@hadoop.apache.org Subject: reducers and data locality Hello folks, I have a lot of input splits (10k-50k - 128 mb blocks) which contains text files. I need to process those line by line, then copy the result into roughly equal size of "shards". So i generate a random key (from

Re: reducers and data locality

-Original Message- From: mete Date: Fri, 27 Apr 2012 09:19:21 To: Reply-To: common-user@hadoop.apache.org Subject: reducers and data locality Hello folks, I have a lot of input splits (10k-50k - 128 mb blocks) which contains text files. I need to process those line by line, then copy the resu