write to most datanode fail quickly

2014-10-14 Thread sunww
HiI'm using hbase with about 20 regionserver. And one regionserver failed to write most of datanodes quickly, finally cause this regionserver die. While other regionserver is ok. logs like this:java.io.IOException: Bad response ERROR for block

Re: write to most datanode fail quickly

2014-10-14 Thread Ted Yu
Which Hadoop release are you using ? Have you run fsck ? Cheers On Oct 14, 2014, at 2:31 AM, sunww spe...@outlook.com wrote: Hi I'm using hbase with about 20 regionserver. And one regionserver failed to write most of datanodes quickly, finally cause this regionserver die. While

RE: write to most datanode fail quickly

2014-10-14 Thread sunww
I'm using Hadoop 2.0.0 and not run fsck. only one regionserver have these dfs logs, strange. Thanks CC: user@hadoop.apache.org From: yuzhih...@gmail.com Subject: Re: write to most datanode fail quickly Date: Tue, 14 Oct 2014 02:43:26 -0700 To: user@hadoop.apache.org Which Hadoop release

Re: write to most datanode fail quickly

2014-10-14 Thread Ted Yu
@hadoop.apache.org From: yuzhih...@gmail.com Subject: Re: write to most datanode fail quickly Date: Tue, 14 Oct 2014 02:43:26 -0700 To: user@hadoop.apache.org Which Hadoop release are you using ? Have you run fsck ? Cheers On Oct 14, 2014, at 2:31 AM, sunww spe...@outlook.com wrote: Hi I'm

RE: write to most datanode fail quickly

2014-10-14 Thread sunww
Hi dfs.client.read.shortcircuit is true. this is namenode log at that moment:http://paste2.org/U0zDA9ms It seems like there is no special in namenode log. Thanks CC: user@hadoop.apache.org From: yuzhih...@gmail.com Subject: Re: write to most datanode fail quickly Date: Tue, 14 Oct 2014 03:09

Re: write to most datanode fail quickly

2014-10-14 Thread Ted Yu
at that moment: http://paste2.org/U0zDA9ms It seems like there is no special in namenode log. Thanks -- CC: user@hadoop.apache.org From: yuzhih...@gmail.com Subject: Re: write to most datanode fail quickly Date: Tue, 14 Oct 2014 03:09:24 -0700 To: user@hadoop.apache.org

RE: write to most datanode fail quickly

2014-10-14 Thread sunww
Hithe correct ip is 132.228.248.20.I check hdfs log in the dead regionserver, it have some error message, maybe it's useful. http://paste2.org/NwpcaGVv Thanks Date: Tue, 14 Oct 2014 10:28:31 -0700 Subject: Re: write to most datanode fail quickly From: yuzhih...@gmail.com To: user