Re: About yarn.scheduler.capacity..disable_preemption

Thanks Sunil. Todd, also, please note that in order to enable the preemption feature, the feature's property must be set to true:yarn.resourcemanager.scheduler.monitor.enable: true Then, if you want to turn preemption off for any particular queue, you would then set yarn.scheduler.capacity..dis

Change log level

Hi All, I compiled the source code, and used eclipse to remotely debug the code, I want to see the Debug information from the log, so I changed the log level for some classes, for example, I changed the FsShell's log level to DEBUG(change it from http://localhost:50070/logLevel), then I add the fo

Re: Eclipse debug HDFS server side code

Thanks a lot. I will explore more related to these. On Tue, Apr 19, 2016 at 2:37 PM, Vinayakumar B wrote: > Usually namenode console logs will be in .out file. > > -vinay > On Apr 20, 2016 12:03 AM, "Kun Ren" wrote: > >> Hi Vinay, >> >> Thanks a lot for your quick reply. >> >> My current confi

Re: Eclipse debug HDFS server side code

Usually namenode console logs will be in .out file. -vinay On Apr 20, 2016 12:03 AM, "Kun Ren" wrote: > Hi Vinay, > > Thanks a lot for your quick reply. > > My current configuration is: I run the pseudo-distributed mode in my one > Linux machine. > Yes, I changed the code in eclipse first, and t

Re: Eclipse debug HDFS server side code

Hi Vinay, Thanks a lot for your quick reply. My current configuration is: I run the pseudo-distributed mode in my one Linux machine. Yes, I changed the code in eclipse first, and then re-compiled the code, and then restart HDFS using the new binary, and re-debug, but still can not see any output

Fwd: Eclipse debug HDFS server side code

-Vinay -- Forwarded message -- From: Vinayakumar B Date: Tue, Apr 19, 2016 at 11:47 PM Subject: Re: Eclipse debug HDFS server side code To: Kun Ren 1. Since you are debugging remote code, you can't change the code dynamically during debugging. If you want to change, put the com

Re: Eclipse debug HDFS server side code

Hi Vinay, Thanks a lot, it works:) I have another question: 1. When I add the following lines into the public boolean mkdirs(String src, FsPermission masked, boolean createParent) in the NameNodeRpcServer.java, LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); System.out.pr

Re: Eclipse debug HDFS server side code

Hi Kun Ren, You can follow the below steps. 1. configure HADOOP_NAMENODE_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in hadoop-env.sh 2. Start Namenode 3. Now Namenode will start debug port in 3988. 4. Configure Remote debug application to connect to :3988 in eclip

Eclipse debug HDFS server side code

Hi All, Currently I used eclipse to compile/debug the source code, and I configured the "remote Java application" to debug the source code in eclipse, for example, I can debug the client side code when I run the command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> DistributedFileSys

Re: About yarn.scheduler.capacity..disable_preemption

Hi Todd If this configuration is enabled for a queue, proportional preemption policy will skip this queue from collecting resources for preemption. And this configuration is enabled for a parent queues, it's also applicable to its child queues as well.(will be inherited) Thanks Sunil On Tue, Apr