Hi,
There are 20 threads which put file into HDFS ceaseless, every file is 2k.
When 1 million files have finished, client begin throw coulod not complete
file exception ceaseless.
At that time, datanode is hang-up.
I think maybe heart beat is lost, so namenode does not know the state of
- Original Message -
From: bourne1900 bourne1...@yahoo.cn
Date: Tuesday, October 18, 2011 3:21 pm
Subject: could not complete file...
To: common-user common-user@hadoop.apache.org
Hi,
There are 20 threads which put file into HDFS ceaseless, every
file is 2k.
When 1 million files
Thank you for your reply.
There is PIPE ERROR in datanode log, and nothing else.
Client only shows Could not complete file ceaselessly.
From namonodeIP:50070/dfshealth.jsp , I found the datanode is hang-up, and
there is only a datanode in my cluster :)
BTW, the retry times is unlimit I think
days then hangs again with the 'Could not complete'
error
in the JobTracker log until we kill the cluster.
2011-09-09 08:04:32,429 INFO org.apache.hadoop.hdfs.DFSClient: Could not
complete file
/log/hadoop/tmp/flow_BYVMTA_family_BYVMTA_72751_8284775/_logs/history
etc.
And it is back.
Confirmed that all the disks setup for HDFS are 'deadline'.
Runs fine for few days then hangs again with the 'Could not complete' error
in the JobTracker log until we kill the cluster.
2011-09-09 08:04:32,429 INFO org.apache.hadoop.hdfs.DFSClient: Could not
complete file
/log
for HDFS are 'deadline'.
Runs fine for few days then hangs again with the 'Could not complete' error
in the JobTracker log until we kill the cluster.
2011-09-09 08:04:32,429 INFO org.apache.hadoop.hdfs.DFSClient: Could not
complete file
/log/hadoop/tmp/flow_BYVMTA_family_BYVMTA_72751_8284775/_logs
.
Confirmed that all the disks setup for HDFS are 'deadline'.
Runs fine for few days then hangs again with the 'Could not complete'
error
in the JobTracker log until we kill the cluster.
2011-09-09 08:04:32,429 INFO org.apache.hadoop.hdfs.DFSClient: Could not
complete file
/log/hadoop
=/10.120.41.105:50010]
11/03/16 09:34:04 INFO hdfs.DFSClient: Abandoning block
blk_2153189599588075377_10039793
11/03/16 09:34:04 INFO hdfs.DFSClient: Waiting to find target node:
10.120.41.105:50010
11/03/16 09:34:55 INFO hdfs.DFSClient: Could not complete file
/tmp/hadoop/mapred/system
org.apache.hadoop.hdfs.DFSClient: Error
Recovery for block blk_3829493505250917008_9959810 in pipeline
10.120.41.105:50010, 10.120.41.102:50010, 10.120.41.103:50010: bad
datanode 10.120.41.103:50010
2011-03-16 02:38:53,133 INFO org.apache.hadoop.hdfs.DFSClient: Could not
complete file
/var/hadoop/tmp
10.120.41.103:50010
2011-03-16 02:38:53,133 INFO org.apache.hadoop.hdfs.DFSClient: Could not
complete file
/var/hadoop/tmp/2_20110316_pmta_pipe_2_20_50351_2503122/_logs/history/hadnn01.atlis1_1299879680612_job_201103111641_0312_deliv_2_20110316_pmta_pipe*2_20110316_%5B%281%2F3%29+...QUEUED_T
You should check the bad reducers' logs carefully.There may be more
information about it.
2011/3/10 Chris Curtin curtin.ch...@gmail.com
Hi,
The last couple of days we have been seeing 10's of thousands of these
errors in the logs:
INFO org.apache.hadoop.hdfs.DFSClient: Could not complete
Hi,
The last couple of days we have been seeing 10's of thousands of these
errors in the logs:
INFO org.apache.hadoop.hdfs.DFSClient: Could not complete file
/offline/working/3/aat/_temporary/_attempt_201103100812_0024_r_03_0/4129371_172307245/part-3
retrying...
When this is going
INFO org.apache.hadoop.mapred.JobInProgress:
Job job_201010140844_0510 has completed successfully.
2010-10-15 04:13:44,192 INFO org.apache.hadoop.hdfs.DFSClient: Could
not complete file
/user/kaduindexer-18509/us/201010150300/dealdocid_pre_merged_1/_logs/hist
ory/phx
13 matches
Mail list logo
