Re: HDFS datanode: rename local filesystem directories?

2021-09-13 Thread Andrew Chi
Thanks, Bob. Comments inline. On Mon, Sep 13, 2021 at 8:06 PM Bob Metelsky wrote: > I think there are some fsck queries you can run where it will show the > full path and then (MISSING) you can find that with google pretty easy. > Think of it… the namenode has to keep track where all the blocks

Re: hadoop-hdfs-native-client Help

2021-09-13 Thread Masatake Iwasaki
Was wondering if you were using an IDE when you ran the Maven commands? I ran commands from terminal right after checking out rel/release-3.3.1 tag with no modification on source tree. I used maven-3.4.5-5 and cmake-3.18.2-11 provided by AppStream repo. # You need newer cmake (and gcc9 and boos

Re: HDFS datanode: rename local filesystem directories?

2021-09-13 Thread Andrew Chi
Thanks a bunch. Most of the data was replicated on other datanodes, but there were some blocks that at the time of failure were only on the single datanode with the failed drives. I did look at the namenode.log, but it seems that for each block, the central log only provides the IP address of the

HDFS datanode: rename local filesystem directories?

2021-09-13 Thread Andrew Chi
I've had a recent drive failure that resulted in the removal of several drives from an HDFS datanode machine (Hadoop version 3.3.0). This caused Linux to rename half of the drives in /dev/*, with the result that when we mount the drives, the original directory mapping no longer exists. The data o