Those files and related data are most probably lost I don't see any
other option than deleting them.
Are you sure those blocks where not missing before the migration? Did you
have any crash over the migration process?
JM
2015-02-03 13:14 GMT-08:00 Ellimilial K ellimil...@googlemail.com:
That's quite horrible, oh well, thanks for the help!
Yes, positive, we started having issues with HA quorum a couple of days
after the migration, HBase has constantly been taking ~200 requests a
second via stargate, things seemed to work fine.
Mateusz
On 3 February 2015 at 22:11, Jean-Marc
We have recently experienced some issues with our namenodes in HA
arrangement and had to recreate namenode metadata from a backup while some
new data has been pushed to the regions ervers in the meantime. We're on
HBase 98.6.
After launching the cluster again, we have realised that we're missing
Hi Mateusz,
Data from this HFile is most probably lost. Is the block also reporting
missing from fsck? Do you have any datanode down which might contain this
block? How big is tis HFile? 929610 bytes only? If so, one option might
just to to delete this HFile.
How many HFiles are within this
Hi Mateusz,
Thats interesting, did you started the NN with the right fsimage after the
upgrade? that might also explain this.
cheers,
esteban.
--
Cloudera, Inc.
On Tue, Feb 3, 2015 at 2:26 PM, Ellimilial K ellimil...@googlemail.com
wrote:
That's quite horrible, oh well, thanks for the
Hi Esteban,
I believe the upgrade went fine, i.e. the stack worked for a couple of days
until the main namenode died yesterday (possibly timed out on gc?) the
backup one died(or did not roll) complaining on out of sync errors from
journalnodes. When I restarted journalnodes both namenodes started
Hi Mateusz,
As JMS mentioned, is very likely the data is lost, but that type of
corruption is usually due some DNs down or data volumes removed for some
reason, have you tried to recover that data from those DNs first?
From for what looks like a continuous stream of regions sounds like you
had a
Thank you for the responses!
@Jean-Mark
This comes from fsck /, I see a flood of those going in at least hundreds,
for this particular region:
/hbase/data/default/table/ffa95306f599dbff99497e71841724fe/extracted/18c428413d7b4a89959911c9112a6eb9:
CORRUPT blockpool
