subject:"Iostat on Hadoop"

Re: Iostat on Hadoop

2011-03-16 Thread Jérôme Thièvre INA

Hi Matthew, you can use iostat -xm 2 to monitor disk usage. Look at %util column. When numbers are between 90-100% for some devices, you start to have some processes that are in disk sleep status and you may have excessive loads. Use htop to monitor disk sleep processes. Sort on the S column and w

Iostat on Hadoop

2011-03-16 Thread Matthew John

Hi all, Can someone give pointers on using Iostat to account for IO overheads (disk read/writes) in a MapReduce job. Matthew John