in 205, code is different than trace Which version are you using?
I just verified the code in older versions,
http://mail-archives.apache.org/mod_mbox/hadoop-common-commits/201109.mbox/%3c20110902221116.d0b192388...@eris.apache.org%3E
below is the code snippet.
+boolean rv = true;
+
+
Hi,
First, you have 8 physical cores. Hyper threading makes the machine think that
it has 16. The trouble is that you really don't have 16 cores so you need to be
a little more conservative.
You don't mention HBase, so I'm going to assume that you don't have it
installed.
So in terms of
You need to create a log directory on your TaskTracker nodes:
/opt/ecip/BMC/hadoopTest/hadoop-0.20.203.0/logs/
Make sure the directory is writable by the mapred user, or which ever
user your TaskTrackers were started as.
-Joey
On Thu, Nov 3, 2011 at 11:11 PM, Li, Yonggang yongga...@hp.com
Hi,
In the jobs running on my cluster of 20 machines, I used to run jobs (via
hadoop jar ...) that would spawn around 4000 map tasks. Now when I run
the same jobs, that number is 20; and I notice that in the job
configuration, the parameter mapred.map.tasks is set to 20, whereas it
never used to
What versions of Hadoop were you running with previously, and what version are
you running with now?
--Bobby Evans
On 11/4/11 9:33 AM, Brendan W. bw8...@gmail.com wrote:
Hi,
In the jobs running on my cluster of 20 machines, I used to run jobs (via
hadoop jar ...) that would spawn around 4000
All the same, no change in that...0.20.2.
Other people do have access to this system to change things like conf
files, but nobody's owning up and I have to figure this out. I have
verified that the mapred.map.tasks property is not getting set in the
mapred-site.xml files on the cluster or in the
In 0.20.2 The JobClient will update mapred.map.tasks to be equal to the number
of splits returned by the InputFormat. The input format will usually take
mapred.map.tasks as a recommendation when deciding on what splits to make.
That is the only place in the code that I could find that is
Plain Java MR, using the Cassandra inputFormat to read out of Cassandra.
Perhaps somebody hacked the inputFormat code on me...
But what's weird is that the parameter mapred.map.tasks didn't appear in
the job confs before at all. Now it does, with a value of 20 (happens to
be the # of machines
I'm trying to run a TeraSort job to confirm that my cluster is set up
correctly. The mappers perform fine, but in the reduce stage all the data
is sent to a single node. My mapred.reduce.tasks parameter is set to an
appropriate value greater than 1. I am launching multiple reducers, but
only one
Could you pastebin the Terasort configuration xml file? I have run Terasort
over 1000 times but I have never seen this problem.
How did you generate the data for terasort? using teragen or some other method?
Raj
Stoser Analytics
www.stoser.com
From: W.P.
Hi there,
I am learning hadoop and looking at the two example Java
codes SecondarySort.java and WordCount.java, using the latest stable
version 0.20.203.0.
One interesting feature I found in the SecondarySort.java code is the use
of the null for the value sent by the reducer.
The code is copied
11 matches
Mail list logo
