[jira] [Resolved] (HBASE-14604) Improve MoveCostFunction in StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-14604.
Resolution: Fixed
Thanks for the patch, Guanghao
> Improve MoveCostFunction in StochasticLoadBalancer
> --
>
> Key: HBASE-14604
> URL: https://issues.apache.org/jira/browse/HBASE-14604
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 0.98.15
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 2.0.0, 1.3.0, 0.98.16
>
> Attachments: HBASE-14604-0.98.patch, HBASE-14604-0.98_v1.patch,
> HBASE-14604-trunk.patch, HBASE-14604-trunk_v1.patch, HBASE-14604_98.diff,
> HBASE-14604_98_with_ut.diff
>
>
> The code in MoveCoseFunction:
> {code}
> return scale(0, cluster.numRegions + META_MOVE_COST_MULT, moveCost);
> {code}
> It uses cluster.numRegions + META_MOVE_COST_MULT as the max value when scale
> moveCost to [0,1]. But this should use maxMoves as the max value when cluster
> have a lot of regions.
> Assume a cluster have 1 regions, maxMoves is 2500, it only scale moveCost
> to [0, 0.25].
> Improve moveCost by use Math.min(cluster.numRegions, maxMoves) as the max
> cost, so it can scale moveCost to [0,1].
> {code}
> return scale(0, Math.min(cluster.numRegions, maxMoves) + META_MOVE_COST_MULT,
> moveCost);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14654) Reenable TestMultiParallel#testActiveThreadsCount
Heng Chen created HBASE-14654: - Summary: Reenable TestMultiParallel#testActiveThreadsCount Key: HBASE-14654 URL: https://issues.apache.org/jira/browse/HBASE-14654 Project: HBase Issue Type: Bug Reporter: Heng Chen It was disabled in HBASE-14642, this issue should reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14655) Narrow the scope of doAs() calls to region observer notifications for compaction
Ted Yu created HBASE-14655: -- Summary: Narrow the scope of doAs() calls to region observer notifications for compaction Key: HBASE-14655 URL: https://issues.apache.org/jira/browse/HBASE-14655 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Attachments: 14655-v1.txt As what has been done in HBASE-14631 and HBASE-14605, the scope of calling doAs() for compaction related region observer notifications should be narrowed. User object is passed from CompactSplitThread down to the methods where region observer notifications are made. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14656) Move TestAssignmentManager from medium to large category
[ https://issues.apache.org/jira/browse/HBASE-14656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-14656. --- Resolution: Fixed Assignee: stack Fix Version/s: 1.3.0 1.2.0 2.0.0 Pushed this to branch-1 and branch-1.2. Test is not in master. > Move TestAssignmentManager from medium to large category > > > Key: HBASE-14656 > URL: https://issues.apache.org/jira/browse/HBASE-14656 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14656.patch > > > On internal rig, this test timedout. Its medium category and then I go this: > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test > (secondPartTestsExecution) on project hbase-server: There are test failures. > Hopefully there is correlation. Let me move this test to large from medium so > it runs in its own jvm... and we see it timeout without breaking next stage > run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14656) Move TestAssignmentManager from medium to large category
stack created HBASE-14656: - Summary: Move TestAssignmentManager from medium to large category Key: HBASE-14656 URL: https://issues.apache.org/jira/browse/HBASE-14656 Project: HBase Issue Type: Sub-task Reporter: stack On internal rig, this test timedout. Its medium category and then I go this: org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (secondPartTestsExecution) on project hbase-server: There are test failures. Hopefully there is correlation. Let me move this test to large from medium so it runs in its own jvm... and we see it timeout without breaking next stage run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14657) Remove unneeded API from EncodedSeeker
Lars Hofhansl created HBASE-14657: - Summary: Remove unneeded API from EncodedSeeker Key: HBASE-14657 URL: https://issues.apache.org/jira/browse/HBASE-14657 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3 See parent. We do not need getKeyValueBuffer. It's only used for tests, and parent patch fixes all tests to use getKeyValue instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14658) Allow loading a ChaosMonkey that isn't one of the set ones.
Elliott Clark created HBASE-14658: - Summary: Allow loading a ChaosMonkey that isn't one of the set ones. Key: HBASE-14658 URL: https://issues.apache.org/jira/browse/HBASE-14658 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Users should be able to define their own chaos monkey that is loaded by class name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14659) Fix flakey TestHFileOutputFormat
Heng Chen created HBASE-14659: - Summary: Fix flakey TestHFileOutputFormat Key: HBASE-14659 URL: https://issues.apache.org/jira/browse/HBASE-14659 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-1521) Optimize codepath for minor compactions
[ https://issues.apache.org/jira/browse/HBASE-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-1521. -- Resolution: Won't Fix Resolving as no longer pertinent. Please reopen if I have it wrong. > Optimize codepath for minor compactions > --- > > Key: HBASE-1521 > URL: https://issues.apache.org/jira/browse/HBASE-1521 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.20.0, 0.90.0 >Reporter: Jonathan Gray >Assignee: Amitanand Aiyer > > There are some additional optimizations in the specialized StoreScanner and > also in HFile for minor compactions. For example, there is some KV > disassembling and sanity checking in HFile even though in a minor compaction > these checks have already been run during the flush. > Another area to discuss is whether we should actually process deletes during > minor compactions. It's not especially expensive (ScanDeleteTracker is quite > simple) but it requires looking at both the row and the qualifier value of > every single KV. Removing this would drop our axiom that "Deletes only apply > to later storefiles", which is used during Get processing to have more > efficient delete handling. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5334) Pluggable Compaction Algorithms
[ https://issues.apache.org/jira/browse/HBASE-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-5334. -- Resolution: Duplicate See above for notes on how this is a duplicate and implemented. > Pluggable Compaction Algorithms > --- > > Key: HBASE-5334 > URL: https://issues.apache.org/jira/browse/HBASE-5334 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Akashnil >Priority: Minor > Labels: compaction, regionserver > > It would be good to create a set of common compaction algorithms so that we > can tune this on a per-CF basis. In order to accomplish this, we need to > refactor the current algorithm for plugability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-9530) Compaction improvements
[ https://issues.apache.org/jira/browse/HBASE-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-9530. -- Resolution: Incomplete Resolving umbrella issue with one subtask and little by way of description > Compaction improvements > --- > > Key: HBASE-9530 > URL: https://issues.apache.org/jira/browse/HBASE-9530 > Project: HBase > Issue Type: Umbrella >Reporter: stack > > Umbrella compaction improvements issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
