Question about mapp Task and reducer Task

. bit1...@163.com

Re: HDFS-based database for Big and Small data?

in your position, I would choose MongoDB with no hesitate bit1...@163.com From: Alec Taylor Date: 2015-01-03 15:44 To: user Subject: HDFS-based database for Big and Small data? Want to replace MongoDB with an HDFS-based database in my architecture. Note that this is a new system, not a rewrite

Re: Multinode setup..

Setting Haddop in real distributed cluster isn't that complex, Which step were you stuck at? bit1...@163.com From: Anil Jagtap Date: 2015-01-03 14:32 To: user Subject: Multinode setup.. Dear All, Im trying to setup a multi node cluster and I found millions of articles on how to configure

Re: Re: Multinode setup..

will be specific, which you can google for answers. bit1...@163.com From: Anil Jagtap Date: 2015-01-03 15:05 To: user@hadoop.apache.org Subject: Re: Multinode setup.. I have a single node already set as per the pdf instructions. I cloned this node and want to set as a slave. How do I do this ? Rgds

Sqoop1.4.5 and Hadoop2.5.2

Hi Hadoopers, I would ask whether Sqoop1.4.5 and Hadoop2.5.2 are comparable, that whether they could work together. Thanks. bit1...@163.com

回复: Question about shuffle/merge/sort phrase

Could some one help me on this question? thanks. bit1...@163.com 发件人： Todd 发送时间： 2014-12-21 21:59 收件人： user@hadoop.apache.org 主题： Question about shuffle/merge/sort phrase Hi, Hadoopers, I got a question about shuffle/sort/merge phrase related.. My understanding is that shuffle is used

Re: RE: Question about shuffle/merge/sort phrase

together. 2. The key/value pairs are sorted by the key 3. All the values of the same key will form an iterative collection into a format like key, value1,value2,value3... My question is who takes this responsibiltiy to form this iterative collection? Thanks. bit1...@163.com From: Rohith

Re: Re: Question about shuffle/merge/sort phrase

Then what exactly happens after Reducer pulls all mapper output key/value pairs from all the mapper nodes before reducer see the key,value1,value2..? bit1...@163.com From: Susheel Kumar Gadalay Date: 2014-12-22 13:20 To: user Subject: Re: Question about shuffle/merge/sort phrase Sorry, typo

A quick question about replication factor

Hi Hadoopers, If I configure the replication factor to be 3 in the configuration file, then how many blocks of the same have been stored? Three or Four. bit1...@163.com

Question about the behavior of HDFS.

most of the data. bit1...@163.com

Re: RE: Question about the behavior of HDFS.

Thanks Natarajan! bit1...@163.com From: Natarajan, Prabakaran 1. (NSN - IN/Bangalore) Date: 2014-12-19 12:22 To: user@hadoop.apache.org Subject: RE: Question about the behavior of HDFS. Where ever you upload, it upload evenly to all machines. Namenode will not have data but has only

Re: Re: Question about the behavior of HDFS.

Thanks Shashwat,but I don't think the paper answers the question, :-). bit1...@163.com From: shashwat shriparv Date: 2014-12-19 12:32 To: firefly...@gmail.com CC: user; bit1129 Subject: Re: Re: Question about the behavior of HDFS. ​Its opening for me any how i am attaching the document

Re: Re: How many blocks does one input split have?

input splits(say, the the original file is 1TB or larger,then there may be thousands of input splits), thousands of mapper tasks would be created. bit1...@163.com From: mark charts Date: 2014-12-18 00:15 To: user@hadoop.apache.org Subject: Re: How many blocks does one input split have? Hello

Re: Re: How many blocks does one input split have?

Sure, thanks Mark. That mean, the completed mapper task is not reused to work on the pending input splits. bit1...@163.com From: daemeon reiydelle Date: 2014-12-18 11:11 To: user CC: mark charts Subject: Re: Re: How many blocks does one input split have? There would be thousands of tasks

Re: Re: Where the output of mappers are saved ?

Thanks Susheel !, understood. bit1...@163.com From: Susheel Kumar Gadalay Date: 2014-12-16 15:27 To: user Subject: Re: Re: Where the output of mappers are saved ? I don't think so. It will be a single output file per reducer. If u want multiple small size output files then specify

Re: Re: Where the output of mappers are saved ?

Do they only exist during the map/reduce process and will be removed after the MR finished？ When the reduce finished，I only see part-m-, part-m-0001 , which are reduce results. bit1...@163.com From: Susheel Kumar Gadalay Date: 2014-12-16 13:05 To: user Subject: Re: Where

Re: Re: Where the output of mappers are saved ?

Thanks Susheel!! One more question.. If part-r- is extremely large,say, 2G, will the file be splitted into more files under the output directory,that is, one reducer could product more than one files. bit1...@163.com From: Susheel Kumar Gadalay Date: 2014-12-16 14:17 To: user Subject