This is one reason to consider virtualizing Hadoop clusters. The idea is to
create multiple virtual clusters on a single physical cluster and apply various
kinds of resource controls (CPU, memory, I/O) on the virtual machines that make
up each virtual cluster. Then if any application or VM within a virtual cluster
crashes, hangs, or tries to hog resources, the other virtual clusters will be
unaffected. Multi-tenancy is also enabled since the isolation between virtual
clusters is secure.
Jeff
----- Original Message -----
From: "Hemanth Yamijala" <yhema...@thoughtworks.com>
To: user@hadoop.apache.org
Sent: Thursday, February 21, 2013 8:51:04 AM
Subject: Re: Hadoop efficient resource isolation
Supporting a multiuser scenario like this is always hard under Hadoop. There
are a few configuration knobs that offer some administrative control and
protection.
Specifically for the problem you describe, you could probably set
Mapreduce.{map|reduce}.child.ulimit on the tasktrackers, so that any job that
is exceeding these limits will be killed. Of course, a side effect of this
would be that jobs would be bound by some limits even if they legitimately
require more memory.
But you could try starting with this.
Thanks
Hemanth
On Thursday, February 21, 2013, Dhanasekaran Anbalagan wrote:
Hi Guys,
It's possible isolation job submission for hadoop cluster, we currently running
48 machine cluster. we monitor Hadoop is not provides efficient resource
isolation. In my case we ran for tech and research pool, When tech job some
memory leak will haven, It's occupy the hole cluster. Finally we figure out
issue with tech job. It's screwed up hole hadoop cluster. finally 10 data node
are dead.
Any prevention of job submission efficient way resource allocation. When
something wrong in particular job, effect particular pool, Not effect others
job. Any way to archive this
Please guide me guys.
My idea is, When tech user submit job means only apply job in for my case
submit 24 machine. other machine only for research user.
It's will prevent the memory leak problem.
-Dhanasekaran.
Did I learn something today? If not, I wasted it.
