Hi, This message is a little long. I beg your patient.
Our team would like to tune MR performance by changing parameters in Hadoop project and JVM according to the MR Job status and result. First, classify MR jobs into several kinds. Then monitor cpu, memory, etc. in a MR job, structing the data from the monitor and input it into HBase. The crucial step is to build a model or models to analysis the data. Finally, acquire the proposal for tuning MR jobs, such as increase the memory for the job or reduce it, etc. However, I am a developer in HBase subproject and not so acquainted with MR jobs. I need some suggestion about the following aspects: * Is this plan feasible or not? why? * Is there any one or team doing the above before? * Which processes in a MR job we ought to monitor more carefully? * Which parameters in that processes we ought to care? * What can we refer for the model building? * Also, any other suggestion about our plan will be welcome. Thank you a lot!!! Evan, 2010-06-02 __________________________________________________ ¸Ï¿ì×¢²áÑÅ»¢³¬´óÈÝÁ¿Ãâ·ÑÓÊÏä? http://cn.mail.yahoo.com
