Name node files are creating different directory

Hi, I configured hadoop 2.7.1 on windows 7 (32 bit ) in C Drive. I tried to format using "*hdfs namenode -forma*t',name node files are created at *C:\tmp\hadoop-user\dfs\name*.but the property "C:\name" in hdfs-site.xml . *hdfs namenode -format* -10.219.149.100-1457674982841 16/03/11 11:13:03

HDFS File Closed Issues

Hi All, I am trying to work with Apache HAWQ with PXF. While trying to write a file in HDFS I am unable to write the files to it. Looking closely at the NameNode logs I find the following messsage: 2016-03-10 19:28:11,759 INFO hdfs.StateChange (FSNamesystem.java:completeFile(3503)) - DIR*

Re: how to use Yarn API to find task/attempt status

On 10 Mar 2016, at 18:21, Frank Luo wrote: Thanks David/Jeff. To avoid further confusions, let me make sure I am clear on what I am trying to do: I would like to know how many hours in a day my cluster is running at its full capacity, and when that happens, how long is my waiting queue. I

Re: how to use Yarn API to find task/attempt status

You would use YARN apis as mentioned my David. Look for “PendingMB” from “RM:8088/jmx” to see allocated/reserved/pending stats on a per queue basis. There is probably a WS that exposes similar data. At the app level, something like

Why localizing takes that long?

I'm running a Hadoop 2.2.0 cluster, I observed a weird latency in launching map task on some nodes for my MR job, by looking at Yarn UI, drill down to the specific task, the UI showed me the task is not started yet, still in "localized state", this will keep like 5-15 minutes, then it can proceed.