Can you restart tasktrackers once and run the job again? It refreshes the
class path.
On Sun, Apr 15, 2012 at 11:58 AM, Bas Hickendorff
wrote:
> Thanks.
>
> The native snappy libraries I have installed. However, I use the
> normal jars that you get when downloading Hadoop, I am not compiling
> Ha
Hey folks,
I have my job tracker GUI which shows a lot of information about the
running/completed jobs.
I am interested in the field "Reduce shuffle bytes". I want to know how it
is computed... Is it just the sum of all the bytes received per reducer
during shuffle ?
Any help?
Thanks
rote:
> 2GB for a task tracker? Here are some possible thoughts.
> Compress map output.
> Change mapred.reduce.slowstart.completed.maps
>
>
> By the way I see no swapping. Anything interesting from the task tracker
> log? System log?
>
> Raj
>
>
>
>
>
&g
Hi Folks,
I am running hive on a 10 node cluster. Since my hive queries have joins in
them, their reduce phases are a bit heavy.
I have 2GB RAM on each TT . The problem is that my reducer hangs at 76% for
a large amount of time. I guess this is due to excessive swapping from disk
to memory. My v
:
> John
>Can you share the hive QL you are using for joins?
>
> Regards
> Bejoy K S
>
> -Original Message-----
> From: john smith
> Date: Mon, 19 Sep 2011 19:02:02
> To:
> Reply-To: common-user@hadoop.apache.org
> Subject: Re: Out of heap space errors o
d.child.java.opts" in mapred-site.xml
>
> by default it will be 200mb. But your io.sort.mb(300) is more than that.
> So, configure more heap space for child tasks.
>
> ex:
> -Xmx512m
>
> Regards,
> Uma
>
> - Original Message -
> From: john smith
> Date
Hey guys,
I am running hive and I am trying to join two tables (2.2GB and 136MB) on a
cluster of 9 nodes (replication = 3)
Hadoop version - 0.20.2
Each data node memory - 2GB
HADOOP_HEAPSIZE - 1000MB
other heap settings are defaults. My hive launches 40 Maptasks and every
task failed with the sa
Hi all,
First of all, ganglia integration with hadoop is an awesome feature. Kudos
to the hadoop devs. Unfortunately its not working out for me. I am unable to
see hadoop specific metrics in my ganglia frontend. My configurations are
as follows:
gmetad.coinf :
data_source "hadoop test"( I a
he network mbufs? Don't know if this
> is your problem as well or not.
>
> --Aaron
> -Original Message-
> From: john smith [mailto:js1987.sm...@gmail.com]
> Sent: Thursday, September 15, 2011 9:46 PM
> To: common-user@hadoop.apache.org
> Subject: Re: Datanodes goi
; heap given number of blocks its gotta report (which would equate to a
> small files issue given your cluster size possibly, but that's a
> different discussion).
>
> On Fri, Sep 16, 2011 at 3:36 AM, john smith
> wrote:
> > Hi all,
> >
> > I am running a 10 n
Hi all,
I am running a 10 node cluster (1NN + 9DN, ubuntu server 10.04, 2GB RAM
each). I am facing a strange problem. My datanodes go down randomly and
nothing showup in the logs. They lose their network connectivity suddenly
and NN declares them as dead. Any one faced this problem? Is it because
are trying to do, but a use-case will help.
>
> IAC, the only way to achieve what you are trying to do is to run to jobs
> with the first a map-only job (i.e. #reduces = 0).
>
> Arun
>
> On Sep 10, 2011, at 10:19 PM, john smith wrote:
>
> > Hey,
> >
> > I have
job with #reduces set to 0.
>
> Arun
>
> On Sep 10, 2011, at 2:06 AM, john smith wrote:
>
> > Hi,
> >
> > Some of the MR jobs I run doesn't need sorting of map-output in each
> > partition. Is there someway I can disable it?
> >
> > Any help?
> >
> > Thanks
> > jS
>
>
Hi,
Some of the MR jobs I run doesn't need sorting of map-output in each
partition. Is there someway I can disable it?
Any help?
Thanks
jS
= hdfs://master
>
> And in mapred-site.xml:
>
> mapred.job.tracker = master:8021
>
> That should do it, so long as the slave hosts can freely access the
> master hosts (no blockage of ports via firewall and such).
>
> On Tue, Sep 6, 2011 at 3:05 PM, john smith wrote:
&g
, john smith wrote:
> Hi Harsh,
>
> My jt log : http://pastebin.com/rXAEeDkC
>
> I have some startup exceptions (which doesn't matter much I guess) but the
> tail indicates that its locating the splits correctly and then it hangs !
>
> Any idea?
>
> Thanks
>
>
k the tail of JobTracker logs after a submit is done to see if
> an error/warn there is causing this. And then dig further on
> why/what/how.
>
> Hard to tell what your problem specifically is without logs :)
>
> On Tue, Sep 6, 2011 at 1:18 PM, john smith wrote:
> > Hi Fo
Hi Folks,
I am working on a 3 node cluster (1 NN + 2 DNs) . I loaded some test data
with replication factor 3 (around 400MB data). However when I run wordcount
example , it hangs at map 0%.
bin/hadoop jar hadoop-examples-0.20.3-SNAPSHOT.jar wordcount /test_data
/out2
11/09/06 13:07:28 INFO input.
Hey folks,
Strangely I get a out of memory exception while building hadoop from source.
I have 2gigs of ram and I've tried building it from both eclipse and
commandline
http://pastebin.com/9pcHg1P9 is the full stack trace. Can anyone help me out
on this?
Thanks,
John Smith
19 matches
Mail list logo
