Thanks a lot Junyong
On Fri, May 11, 2012 at 11:15 AM, JunYong Li wrote:
> Each has its own merits.
> http://developer.yahoo.com/hadoop/tutorial/module7.html#monitoring
>
> 2012/5/11 Manu S
>
> > Hi All,
> >
> > Which is the best monitoring tool for Hadoop cluster monitoring? Ganglia
> or
> > N
i have encounter CLassNotFound mant times and the reason is the class dose
really not exits in the classpath.
the computer can't go wrong, so calm down, look it over
2012/5/11 Pat Ferrel
> Thanks, that's where I started but #3, which has several methods I tried,
> did not work and the reference
The client is done writing data to the given filename, and would like to
complete it. The function returns whether the file has been closed
successfully. If the function returns false, the caller should try again. A
call to complete() will not return true until all the file's blocks have
been repli
Each has its own merits.
http://developer.yahoo.com/hadoop/tutorial/module7.html#monitoring
2012/5/11 Manu S
> Hi All,
>
> Which is the best monitoring tool for Hadoop cluster monitoring? Ganglia or
> Nagios?
>
> Thanks,
> Manu S
>
--
Regards
Junyong
Cool :) Thanks for your quick reply.
On Fri, May 11, 2012 at 11:07 AM, v v wrote:
> Ganglia won't send alerts, but it is a very good monitoring tool for
> viewing the metrics (hadoop has support for emitting metrics to
> ganglia).
>
> If you need alerting (eg, dfs namespace > 90%) then you will
Hello,
I am running the job, when in two similar size clusters, in one it is running
fine on other gives fallowing exceptions.
I checked the safe mode is off.
Not sure what do I need fix to avoid this?
2012-05-10 00:38:02,637 ERROR org.apache.hadoop.hdfs.DFSClient: Exception
closing file /us
thanks, let me run more of this with the settings provided later in
this thread and provide the details
On Wed, May 9, 2012 at 10:12 PM, Harsh J wrote:
> Can you share your job details (or a sample reducer code) and also
> share your exact error?
>
> If you are holding reducer provided values/key
thanks, let me try this
On Wed, May 9, 2012 at 11:27 PM, Zizon Qiu wrote:
> try setting a lower value for mapred.job.shuffle.input.buffer.percent .
> the reducer used it to decide whether use in-memory shuffle.
> the default value is 0.7,meaning 70% of the "memory" are used as shuffle
> buffer.
Darrell
Are the new dn,nn and mapred directories on the same physical disk? Nothing on
NFS , correct?
Could you be having some hardware issue? Any clue in /var/log/messages or dmesg?
A non responsive system indicates a CPU that is really busy either doing
something or waiting for something and
Thanks, that's where I started but #3, which has several methods I
tried, did not work and the reference claims they are not currently
supported in cloudera's implementation. Not sure why so I don't know if
it's a hadoop thing or not.
I ended up putting the classes in the job jar (#2 of cloude
If the analysis you mention is to create "view" of multiple tables. Once
your data is sorted by the keys in HDFS. You could try Map Side join or
Reducer Side join in Hadoop to generate the "view" of your data (same
keys of multiple data sets are combined). There are many code samples
web, play
Hi，all：
Now I want to setup the security with hbase by kerberos.
As I know, the hbase's ugi is based on the hadoop UserGroupInformation
without parameter "hadoop.job.ugi" after 0.20.2. So when I use the cdh3u3,
the ugi can be generated by two authentication : simple or kerberos.
Firstly
I am more worried about the analysis assuming this data is in HDFS.
-Original Message-
From: Shi Yu [mailto:sh...@uchicago.edu]
Sent: 10 May 2012 18:58
To: common-user@hadoop.apache.org
Subject: RE: SQL analysis
Flume might be suitable for your case.
https://cwiki.apache.org/FLUME/
Flume might be suitable for your case.
https://cwiki.apache.org/FLUME/
Shi
Our focus as of now is on batch queries. And keen to explore on the approach to
get to the path analysis.
Regards,
Karanveer
-Original Message-
From: Shi Yu [mailto:sh...@uchicago.edu]
Sent: 10 May 2012 17:02
To: common-user@hadoop.apache.org
Subject: Re: SQL analysis
It depends on y
It depends on your use case, for example, query only or you have
requirement of real time insert and update. The solutions can
be different.
You might need consider HBase, Cassandra or tools like Flume.
On Thursday 10 May 2012 04:25 PM, Devaraj k wrote:
Hi Tousif,
You can kill the Running Job using the killJob() client API.
If you want to kill the job itself, you can get the job id using task attempt
id from map() or reduce() functions, and you can invoke the killJob() API based
on you
Hi Tousif,
You can kill the Running Job using the killJob() client API.
If you want to kill the job itself, you can get the job id using task attempt
id from map() or reduce() functions, and you can invoke the killJob() API based
on your condition.
Thanks
Devaraj
__
On Thu, May 10, 2012 at 9:33 AM, Todd Lipcon wrote:
> That's real weird..
>
> If you can reproduce this after a reboot, I'd recommend letting the DN
> run for a minute, and then capturing a "jstack " as well as
> the output of "top -H -p -b -n 5" and send it to the list.
What I did after the r
Actually, the ChecksumError happens every time, but it can happen at
different points of the execution...sometimes at the beginning, and
sometimes at the tail end of the map phase.
Hoping to hear from someone with a workaround...
Regards,
Safdar
On Thu, May 10, 2012 at 7:59 AM, Ali Safdar Kurei
Hi,
i have scheduled a map reduce job and i want this map reduce job to kill
itself upon some condition. overall it should fail. Here I'm not using
run method of tool runner.
Fourie,
Good to know. Just for the record, do you remember the version of
OpenJDK that was installed? So that we can mark that as unusable in
our docs/wiki.
On Thu, May 10, 2012 at 12:22 PM, Fourie Joubert
wrote:
> Hi
>
> Yes - that was indeed the problem...
>
> I cleaned up the Java's on all th
That's real weird..
If you can reproduce this after a reboot, I'd recommend letting the DN
run for a minute, and then capturing a "jstack " as well as
the output of "top -H -p -b -n 5" and send it to the list.
What JVM/JDK are you using? What OS version?
-Todd
On Wed, May 9, 2012 at 11:57 PM,
We are looking at doing some initial analysis on SQL text info within the query
runs to come up with some kind of path output to depict how various tables are
linked to each other. For example. A 'view' might be a join from two table's
top of the hierarchy and in turn might be creating some new
24 matches
Mail list logo
