Hi
I need to use hadoop-tool-kit for monitoring. So I followed
http://code.google.com/p/hadoop-toolkit/source/checkout
and applied the patch in my hadoop.20.2 directory as: patch -p0 < patch.20.2
and set a property *“mapred.performance.diagnose”* to true in *
mapred-site.xml*.
but I do
Sorry for the spam, but I didn't see my previous email yet.
I need to use hadoop-tool-kit for monitoring. So I followed
http://code.google.com/p/hadoop-toolkit/source/checkout
and applied the patch in my hadoop.20.2 directory as: patch -p0 < patch.20.2
and set a
On May 17, 2011, at 1:01 PM, Mark question wrote:
> Hi
>
> I need to use hadoop-tool-kit for monitoring. So I followed
> http://code.google.com/p/hadoop-toolkit/source/checkout
>
> and applied the patch in my hadoop.20.2 directory as: patch -p0 < patch.20.2
Lo
e:
>
> > Hi
> >
> > I need to use hadoop-tool-kit for monitoring. So I followed
> > http://code.google.com/p/hadoop-toolkit/source/checkout
> >
> > and applied the patch in my hadoop.20.2 directory as: patch -p0 <
> patch.20.2
>
> Looking a
On May 17, 2011, at 3:11 PM, Mark question wrote:
> So what other memory consumption tools do you suggest? I don't want to do it
> manually and dump statistics into file because IO will affect performance
> too.
We watch memory with Ganglia. We also tune our systems such that a
task wi
Also, it seems like Ganglia would be very well complemented by Nagios
to allow you to monitor an overall health of your cluster.
--
Take care,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do
Thanks for the inputs, but I'm running on a university cluster, not my own
and hence are the assumptions such as each task(mapper/reduer) will take 1
GB valid ?
So I guess to tune performance I should try running the job multiple times
and rely on execution time as an indicator of success.
Thank
