0.20.203 fixed the TT to be more aggressive about heartbeats, but not overtly
so - that should help a lot.
Arun
On Aug 18, 2011, at 2:25 PM, Mck wrote:
>
>> How long are your tasks, and which version of Hadoop are you using?
>
> Hadoop-0.20.1 (eventually we're looking to upgrade to Brisk).
>
Okay, great!
Unfortunately the task launching is still slow even if you have JVM reuse set
to -1 because of heartbeats (the slave node only updates its state with the
master every ~5 seconds).
Matei
On Aug 18, 2011, at 5:25 PM, Mck wrote:
>
>> How long are your tasks, and which version of Ha
> How long are your tasks, and which version of Hadoop are you using?
Hadoop-0.20.1 (eventually we're looking to upgrade to Brisk).
Tasks take 5-30 seconds.
> In older versions (0.20.*), the fair
> scheduler doesn't launch multiple tasks per heartbeat, so it performs poorly
> when your tasks a
How long are your tasks, and which version of Hadoop are you using? In older
versions (0.20.*), the fair scheduler doesn't launch multiple tasks per
heartbeat, so it performs poorly when your tasks are small (less than 5-10
seconds). You may be able to improve it a bit by setting
mapred.fairsch
I'm new to setting up hadoop's scheduler and i'm trying to set up
Fairscheduler on a 3-node cluster. The initial setup is fine but
throughput is abysmal.
Each node is configured with 16 map task capacity and 8 reduce task
capacity. Most jobs being run are reading data from cassandra installed
on t
