Sorry, I also attached a photo of the job Counters but I guess it was filtered.
So the job counter shows:
Reduce output records0 4 4
Spilled Records9 918<--eliminate this for now
Map output records 9 0 9
Reduce input records 0
Ok got this point, thanks Harsh. But my experiment now is to eliminate # of
spilled records for this small light job.
This part of the map log:
2011-02-26 16:05:35,307 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100
2011-02-26 16:05:35,467 INFO org.apache.hadoop.mapred.MapTask: data buffe
Hello,
On Sun, Feb 27, 2011 at 9:30 AM, maha wrote:
> 2011-02-26 16:05:35,571 INFO org.apache.hadoop.mapred.MapTask: Finished spill
> 0 <--- WHY IS THIS ZERO WHEN FINAL JOB COUNTER SAYS
> IT'S 9 SPILLED RECORDS FROM MAPS!
That isn't a record-based count. It is the counter
Hi,
Even though I'm running a very simple job of 4 small records (2 per map) and
one reducer .. I still get all 9 records output of map spilled.
But map-logs shows spilled to be ZERO !
2011-02-26 16:05:35,301 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1
2011-02-26 16:05:35,307 INFO
On 02/26/2011 02:27 PM, MANISH SINGLA wrote:
actually I have kmeansMapper,reducer and driver
files...seperately...and now I want to execute them on the
terminal...using Hadoop...what are the steps that I should
follow...e.g.when running the wordcount program...its simply preseent
in the hadoop-ex
On Sat, Feb 26, 2011 at 10:35 PM, Hari Sreekumar
wrote:
> I have cygwin. But how can the unix shell commands run in windows through
> eclipse?
Cygwin takes care of that. All you have to do is add your
{CygwinsHome}/bin/ to the Windows PATH env-var, and restart your
normal Windows version Eclipse.
I have cygwin. But how can the unix shell commands run in windows through
eclipse? I tried running eclipse through cygwin and then building. In this
case, it does not recognize the path:
BUILD FAILED
C:\Users\Hari\workspace\hadoop-common-trunk\build.xml:332: Unable to find a
javac compiler;
com.su
Hello,
On Sat, Feb 26, 2011 at 9:48 PM, Hari Sreekumar
wrote:
> Is it possible to set up hadoop in eclipse in windows only for browsing code
> without using cygwin? I see some unix specific commands being executed in
> the eclipse target of the build.xml file (tr and sh commands are being
> calle
Is it possible to set up hadoop in eclipse in windows only for browsing code
without using cygwin? I see some unix specific commands being executed in
the eclipse target of the build.xml file (tr and sh commands are being
called at some points) . Can I just comment them out and will it still work
f
Thanks James. Well all the config. files and shared keys are on a shared
storage that is accessed by all the nodes in the cluster.
At times, everything runs fine on initialization, but at other times, the
same problem persists, so was bit confused.
Also, checked the TaskTracker logs on those nodes,
Maybe your ssh keys aren’t distributed the same on each machine or the machines
aren’t configured the same?
J
On 2011-02-26, at 8:25 AM, bikash sharma wrote:
> Hi,
> I have a 10 nodes Hadoop cluster, where I am running some benchmarks for
> experiments.
> Surprisingly, when I initialize the Ha
Hi,
I have a 10 nodes Hadoop cluster, where I am running some benchmarks for
experiments.
Surprisingly, when I initialize the Hadoop cluster
(hadoop/bin/start-mapred.sh), in many instances, only some nodes have
TaskTracker process up (seen using jps), while other nodes do not have
TaskTrackers. Cou
actually I have kmeansMapper,reducer and driver
files...seperately...and now I want to execute them on the
terminal...using Hadoop...what are the steps that I should
follow...e.g.when running the wordcount program...its simply preseent
in the hadoop-examples.jar...but kmeans is not...so I have take
Hi,
> I m facing a problem...I m not able to run the kmeans clustering algo
> on a singls node...till now I have just run the wordcount
> program...what are the steps in doing so???
please give more information on problem you are facing (error, log entries etc).
--
Thanks,
M. Varadharajan
14 matches
Mail list logo
