I am not sure why it can't find the block, If I go to the master I am able
to view the file just fine.
11/02/05 10:41:27 INFO hdfs.DFSClient: Could not obtain block
blk_-6466695353427601952_10797526 from any node: java.io.IOException: No
live nodes contain current block
hadoop fsck /user/felix/120thslice.data -files -blocks -locations
/user/felix/120thslice.data 899130927 bytes, 7 block(s): OK
0. blk_-6466695353427601952_10797526 len=134217728 repl=2 [
192.168.133.37:50010, 192.168.133.41:50010]
1. blk_1491166185638671705_10797526 len=134217728 repl=2 [
192.168.133.13:50010, 192.168.133.10:50010]
2. blk_8579051870161779940_10797526 len=134217728 repl=2 [
192.168.133.29:50010, 192.168.133.3:50010]
3. blk_-5745283133248815034_10797526 len=134217728 repl=2 [
192.168.133.43:50010, 192.168.133.9:50010]
4. blk_-3669078911867988489_10797526 len=134217728 repl=2 [
192.168.133.35:50010, 192.168.133.2:50010]
5. blk_3062577211258193990_10797526 len=134217728 repl=2 [
192.168.133.13:50010, 192.168.133.6:50010]
6. blk_2678797396720686402_10797526 len=93824559 repl=2 [
192.168.133.11:50010, 192.168.133.12:50010]
Status: HEALTHY
Total size: 899130927 B
Total dirs: 0
Total files: 1
Total blocks (validated): 7 (avg. block size 128447275 B)
Minimally replicated blocks: 7 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 2.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 33
Number of racks: 1
when I am tailing the datanode log on both datanodes that host this block
with nothing shows up in the log, it seems the data is not routed to the
datanode at all.
tail -f hadoop-hadoop-datanode-xxxx.log | grep
blk_-6466695353427601952_10797526
here is my code snippet
Configuration conf = new Configuration();
conf.set("fs.default.name", fsname);
int maxXceiverCount = conf.getInt("dfs.datanode.max.xcievers"
,MAX_XCEIVER_COUNT);
int maxIdelTime = conf.getInt("ipc.client.connection.maxidletime",-1);
System.out.println("maxXceiverCount:"+maxXceiverCount+",
maxIdelTime:"+maxIdelTime);
conf.set("ipc.client.connect.max.retries", "5"); //<-- this currently does
not work
try {
hdfs_ref = FileSystem.get(conf);
} catch (IOException e) {
e.printStackTrace();
return;
}
if (hdfs_ref != null) {
Path p = new Path(path);
FileStatus[] status = hdfs_ref.globStatus(p);
System.out.println("Found "+status.length+" files matches pattern");
for (FileStatus s : status){
fis = hdfs_ref.open(s.getPath());
BufferedReader reader = new
BufferedReader(newInputStreamReader(fis));
while(reader.ready()){
String line = reader.readLine();
System.out.println(s.getPath()+":"+line);
}
reader.close();
fis.close();
}
}
Thanks in advance.
Felix
