Hi Seonyoung!
Please take a look at this file :
https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java#L208
.
This is an auxiliary service that runs inside the
Hi all,
We've run a hadoop cluster (Apache Hadoop 2.7.1) with 40 datanodes.
Currently, we're using Fair Scheduler in our cluster.
And there are no limits on the number of concurrent running jobs.
30 ~ 50 I/O heavy jobs has been running concurrently at dawn.
Recently we got shuffle errors as
