>    When you stripe you automatically make every disk in the system have the
> same speed as the slowest disk.  In our experiences, systems are more likely
> to have a 'slow' disk than a dead one.... and detecting that is really
> really hard.  In a distributed system, that multiplier effect can have
> significant consequences on the whole grids performance.

All disk are the same, so there is no speed difference. Stripping can speed up
disk reads (4x more heads to read at the same time).

>    If you actually have all 40 task slots going, the system is likely
> spending quite a bit of time paging...

interestingly, the swap usage is 0. Looks like my tasks are not
utilizing all alloted
heap.

>    We generally use cores/2-1 for map and reduce slots.  This leaves some
> cores and memory for the OS to use for monitoring, data node, etc. So 8
> cores=3 maps, 3 reduces per node.  Going above that has been done for
> extremely lightweight processes, but you're more likely headed for heartache
> if you aren't careful.

I will try to reduce number of tasks per your recommendation to see if
it would make
any significant difference.

Thanks you for all suggestions.

Vadim

Reply via email to