How to see total pending containers ?

Hi, Is there a way to see total pending containers in a cluster,so that we know how far behind we are with etl ? There is a pending containers field on the scheduler page under dr. who table,but that is always zero. -- Thanks, Ashwin

Why resource requests are normalized in RM ?

Hi, Anyone knows why resource requests from AMs are normalized to be multiples of yarn.scheduler.minimum-allocation-mb which is 1G by default ? Also is there any problem with reducing yarn.scheduler.minimum-allocation-mb to less than 1G ? /** * Utility method to normalize a list of resource

assignMultiple and continuous scheduling in Fair scheduler

Hi, I see these two knobs in Fair scheduler - 'assignMultiple' and 'continuos scheduling'. 1. Are there performance benefits using them ? What are the cons ? 2. Also is there any problem with 'continuous scheduling',I'm asking because this is not mentioned in the FS doc ? -- Thanks, Ashwin

Re: Differences between HistoryServer and Yarn TimeLine server?

this issue. On Wed, Apr 23, 2014 at 11:25 PM, Ashwin Shankar ashwinshanka...@gmail.com wrote: Hi Zhijie, There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can you please let me know what is the difference ? Is timeline server finally going to be YARN321+YARN1530 ? You mentioned

Re: Differences between HistoryServer and Yarn TimeLine server?

Hi Zhijie, There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can you please let me know what is the difference ? Is timeline server finally going to be YARN321+YARN1530 ? You mentioned that MR is going to integrated with timeline server,is there a jira I can watch ? Thanks,

Re: Setting debug log level for individual daemons

at 2:06 AM, Ashwin Shankar ashwinshanka...@gmail.com wrote: Thanks Gordon and Stanley, but this would require us to bounce the process. Is there a way to change log levels without bouncing the process ? On Tue, Apr 15, 2014 at 3:23 AM, Gordon Wang gw...@gopivotal.com wrote: Put

Re: Setting debug log level for individual daemons

. log4j.logger.org.apache.hadoop.yarn.server.resourcemanager=DEBUG,console On Tue, Apr 15, 2014 at 8:33 AM, Ashwin Shankar ashwinshanka...@gmail.com wrote: Hi, How do we set log level to debug for lets say only Resource manager and not the other hadoop daemons ? -- Thanks, Ashwin -- Regards Gordon Wang

Setting debug log level for individual daemons

Hi, How do we set log level to debug for lets say only Resource manager and not the other hadoop daemons ? -- Thanks, Ashwin

Resetting dead datanodes list

Hi, Hadoop-1's name node UI displays dead datanodes even if those instances are terminated and are not part of the cluster anymore. Is there a way to reset the dead datenode list without bouncing namenode ? This would help me in my script(which would run nightly) which parses the html

Job fails if I change HADOOP_USER_NAME

Hi, I'm writing a new feature in Fair scheduler and wanted to test it out by running jobs submitted by different users from my laptop. My sleep job runs fine as long as the user name is my mac user name. If I change my hadoop user name by setting HADOOP_USER_NAME, my jobs fail with the exception

Re: Job fails if I change HADOOP_USER_NAME

: Hi The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.? Thanks Regards Rohith Sharma K S *From:* Ashwin Shankar [mailto:ashwinshanka...@gmail.com] *Sent:* 21 March 2014 14:02 *To:* user