Hi,
I want to custom FileInputFormat.class. In order to determine which host
the specific part of a file belongs to, I need to open the file in HDFS and
read some information. It will take me nearly 500ms to open a file and get
the information I need. But now I have thousands of files to deal
Can you expand on your use case a little bit please? It may be that you're
duplicating functionality.
You can take a look at the CombineFileInputFormat for inspiration. If this
is indeed taking a long time, one cheap to implement thing you can do is to
parallelize the calls to get block
Hi,
I want to setup 2-node HDFS: a master NameNode and a Checkpoint
NameNode. My configuration is pretty standard (mostly the default).
//on the 1st machine
#hdfs namenode
//on the 2nd machine
#hdfs namenode -checkpoint
After the startup, the Checkpoint NameNode repeatedly print out the
I am curious about this, too.
On Sat, Nov 29, 2014 at 2:35 PM, Alice 6900848...@gmail.com wrote:
Hi,all:
Namenode HA (NFS, QJM) is available in hadoop 2.x (HDFS-1623). It provides
fast failover for Namenode, but I can't find any description on how long
does it take to recover from failure.
