For your first question, region server web UI,
rs-status#regionRequestStats, shows Write Request Count.
You can monitor the value for the underlying region to see if it receives
above-normal writes.
Cheers
On Mon, Nov 10, 2014 at 4:06 PM, Brian Jeltema bdjelt...@gmail.com wrote:
Was the
Thanks. I appear to have resolved this problem by restarting the HBase Master
and the RegionServers
that were reporting the failure.
Brian
On Nov 11, 2014, at 12:13 PM, Ted Yu yuzhih...@gmail.com wrote:
For your first question, region server web UI,
rs-status#regionRequestStats, shows Write
the checkResource Ted mentioned is a good suspect. see online hbase book
9.7.7.7.1.1. Being Stuck.
Did you see below message in your RS log?
LOG.info(Waited + (System.currentTimeMillis() - fqe.createTime) +
ms on a compaction to clean up 'too many store files'; waited +
or:
LOG.warn(Region + region.getRegionNameAsString() + has too
many +
store files; delaying flush up to + this.blockingWaitTime +
ms);
sth like:
WARN org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Region
I’m running a map/reduce job against a table that is performing a large number
of writes (probably updating every row).
The job is failing with the exception below. This is a solid failure; it dies
at the same point in the application,
and at the same row in the table. So I doubt it’s a conflict
There could be more than one reason where RegionTooBusyException is thrown.
Below are two (from HRegion):
* We throw RegionTooBusyException if above memstore limit
* and expect client to retry using some kind of backoff
*/
private void checkResources()
* Try to acquire a lock.
How many tasks may write to this row concurrently ?
only 1 mapper should be writing to this row. Is there a way to check which
locks are being held?
Which 0.98 release are you using ?
0.98.0.2.1.2.1-471-hadoop2
Thanks
Brian
On Nov 10, 2014, at 2:21 PM, Ted Yu yuzhih...@gmail.com wrote:
Was the region containing this row hot around the time of failure ?
Can you check region server log (along with monitoring tool) what memstore
pressure was ?
Thanks
On Nov 10, 2014, at 11:34 AM, Brian Jeltema brian.jelt...@digitalenvoy.net
wrote:
How many tasks may write to this row
Was the region containing this row hot around the time of failure ?
How do I measure that?
Can you check region server log (along with monitoring tool) what memstore
pressure was ?
I didn't see anything in the region server logs to indicate a problem. And
given the
reproducibility of
