.org]
Sent: Tuesday, May 21, 2013 12:59 PM
To: user@hadoop.apache.org
Subject: Re: Shuffle phase replication factor
The map output doesn't get written to HDFS. The map task writes its output to
its local disk, the reduce tasks will pull the data through HTTP for further
processing.
Am 21.05
The map output doesn't get written to HDFS. The map task writes its output to
its local disk, the reduce tasks will pull the data through HTTP for further
processing.
Am 21.05.2013 um 19:57 schrieb John Lilley :
> When MapReduce enters “shuffle” to partition the tuples, I am assuming that
> it
Intermediate data is written to local disk, not to HDFS.
Ian.
On May 21, 2013, at 1:57 PM, John Lilley wrote:
> When MapReduce enters “shuffle” to partition the tuples, I am assuming that
> it writes intermediate data to HDFS. What replication factor is used for
> those temporary files?
> jo
[mailto:k...@123.org]
Sent: Tuesday, May 21, 2013 12:59 PM
To: user@hadoop.apache.org
Subject: Re: Shuffle phase replication factor
The map output doesn't get written to HDFS. The map task writes its output to
its local disk, the reduce tasks will pull the data through HTTP for further
processing
code.
>
> john
>
> ** **
>
> *From:* Kai Voigt [mailto:k...@123.org]
> *Sent:* Tuesday, May 21, 2013 12:59 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Shuffle phase replication factor
>
> ** **
>
> The map output doesn't get written to HDFS. The map tas
l simply robust enough to allow
a server-side to disconnect at any time to free up slots and the client-side
will retry the request?
Thanks
john
From: Shahab Yunus [mailto:shahab.yu...@gmail.com]
Sent: Wednesday, May 22, 2013 8:38 AM
To: user@hadoop.apache.org
Subject: Re: Shuffle phase replica
ide will retry the request?
>
> Thanks
>
> john
>
> ** **
>
> *From:* Shahab Yunus [mailto:shahab.yu...@gmail.com]
> *Sent:* Wednesday, May 22, 2013 8:38 AM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Shuffle phase replication factor
>
>
pending/failing connection attempts that exceed the limit?
Thanks!
john
From: Rahul Bhattacharjee [mailto:rahul.rec@gmail.com]
Sent: Wednesday, May 22, 2013 8:52 AM
To: user@hadoop.apache.org
Subject: Re: Shuffle phase replication factor
There are properties/configuration to control the no. of
the pending/failing connection attempts that exceed the
> limit?
>
> Thanks!
>
> john
>
> ** **
>
> *From:* Rahul Bhattacharjee [mailto:rahul.rec@gmail.com]
> *Sent:* Wednesday, May 22, 2013 8:52 AM
>
> *To:* user@hadoop.apache.org
> *Subje
?
Thanks,
John
From: erlv5...@gmail.com [mailto:erlv5...@gmail.com] On Behalf Of Kun Ling
Sent: Wednesday, May 22, 2013 7:50 PM
To: user
Subject: Re: Shuffle phase replication factor
Hi John,
1. for the number of simultaneous connection limitations. You can configure
this using the
ask? Or something more persistent in MapReduce?
>
>
> Thanks,
>
> John
>
> ** **
>
> *From:* erlv5...@gmail.com [mailto:erlv5...@gmail.com] *On Behalf Of *Kun
> Ling
> *Sent:* Wednesday, May 22, 2013 7:50 PM
> *To:* user
>
> *Subject:* Re: Shuffle phase
11 matches
Mail list logo