[
https://issues.apache.org/jira/browse/HBASE-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715858#action_12715858
]
ryan rawson commented on HBASE-1477:
------------------------------------
we need to revisit after HBASE-1304 - scans are getting a whole lot faster...
> Contention on META stalls mapreduce job execution
> -------------------------------------------------
>
> Key: HBASE-1477
> URL: https://issues.apache.org/jira/browse/HBASE-1477
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
>
> From Jeremy Pinkham up on hbase-users@:
> bq. A typical mapper in the job takes several minutes, how many minutes
> depends on whether I use the the region partitioner and how many I let run
> concurrently... it's been anywhere from 2 minutes with no partitioner and
> small concurrency (5 mappers) to 8 minutes with the region partitioner and
> high concurrency (150 mappers). This seems to directly correlate with how
> long it takes to do a simple count of .META. while each job is running (2
> seconds to 1 minute)
> bq. I was able to get past this issue affecting my data load by reorganizing
> some of my workflow and data structures to force the ordering of keys without
> the region partitioner. Those changes appear to have side stepped the
> problem for me as I can now load from 100+ mappers without seeing the
> degradation that I was seeing with 40 when using the partitioner (and getting
> some sweet numbers in the requests column of the UI). It's still an
> interesting scaling situation with the region partitioner, but I'm good to go
> without it.
> I have seen this also in the form of freezing of master UI during high load,
> where the UI comes back as soon as load is reduced. When I thread dump it
> looks like all IPC handlers on the region server hosting .META. are busy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
