Re: Some block can not be found while inserting data into HBase

2014-05-28 Thread Jean-Marc Spaggiari
Directly form HDFS, can you try to get the file /apps/hbase/data/WALs/ b08.jsepc.com,60020,1400569539507/b08.jsepc.com%2C60020%2C1400569539507.1401178568958 ? I suspect there is an issue on the HDFS side for this file... 2014-05-28 13:39 GMT-04:00 Ted Yu : > Can you check the audit log on the n

Re: Some block can not be found while inserting data into HBase

2014-05-28 Thread Ted Yu
Can you check the audit log on the namenode to see whether /apps/hbase/data/WALs/ b08.jsepc.com,60020,1400569539507/b08.jsepc.com%2C60020%2C1400569539507.1401178572707 (just an example) was deleted by some client ? If there is an entry for the file in audit log, you should be able to trace to the

Re: Some block can not be found while inserting data into HBase

2014-05-27 Thread Tao Xiao
fsck on /apps/hbase says that it is healthy 2014-05-28 10:58 GMT+08:00 Bharath Vissapragada : > Run an fsck on /hbase to check if there are any inconsistencies. > > > On Wed, May 28, 2014 at 6:23 AM, Tao Xiao > wrote: > > > I‘m using HDP 2.0.6 > > > > > > 2014-05-28 0:03 GMT+08:00 Ted Yu : > >

Re: Some block can not be found while inserting data into HBase

2014-05-27 Thread Bharath Vissapragada
Run an fsck on /hbase to check if there are any inconsistencies. On Wed, May 28, 2014 at 6:23 AM, Tao Xiao wrote: > I‘m using HDP 2.0.6 > > > 2014-05-28 0:03 GMT+08:00 Ted Yu : > > > What hbase / hadoop release are you using ? > > > > Cheers > > > > > > On Tue, May 27, 2014 at 4:25 AM, Tao Xiao

Re: Some block can not be found while inserting data into HBase

2014-05-27 Thread Tao Xiao
I‘m using HDP 2.0.6 2014-05-28 0:03 GMT+08:00 Ted Yu : > What hbase / hadoop release are you using ? > > Cheers > > > On Tue, May 27, 2014 at 4:25 AM, Tao Xiao > wrote: > > > I put massive records into HBase and found that one of the region servers > > crashed. I checked the RS log and NameNode

Re: Some block can not be found while inserting data into HBase

2014-05-27 Thread Ted Yu
What hbase / hadoop release are you using ? Cheers On Tue, May 27, 2014 at 4:25 AM, Tao Xiao wrote: > I put massive records into HBase and found that one of the region servers > crashed. I checked the RS log and NameNode log and found them complaining > that some block does not exist. > > For

Some block can not be found while inserting data into HBase

2014-05-27 Thread Tao Xiao
I put massive records into HBase and found that one of the region servers crashed. I checked the RS log and NameNode log and found them complaining that some block does not exist. For example: *In RS's log:* java.io.IOException: Bad response ERROR for block BP-898918553-10.134.101.112-13939048986