[jira] [Commented] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156571#comment-16156571
]
stack commented on HBASE-14997:
---
Follow-up is HBASE-18769 to move over CompareFilter to new generic
CompareOperator
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch,
> HBASE-14997.master.002.patch, HBASE-14997.master.003.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18769) Make CompareFilter use generic CompareOperator instead of internal enum
stack created HBASE-18769: - Summary: Make CompareFilter use generic CompareOperator instead of internal enum Key: HBASE-18769 URL: https://issues.apache.org/jira/browse/HBASE-18769 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0-alpha-3 Should probably do this so only the one enum in the codebase. Means a bunch of boilerplate deprecating the old enum for the new. Suggested by [~anoop.hbase]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156570#comment-16156570
]
stack commented on HBASE-14997:
---
.003 incorporates [~anoop.hbase] review. He points out we break Region
Interface and CPs. He is right. On the former, we've already broken it by
removing methods and adding new (non-default) implementations. On latter, CPs
are going to be incompatible in hbase2. He also suggests that we use new
generic compare operator even inside the CompareFilter. This latter is probably
worth doing. Filing a follow-up.
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch,
> HBASE-14997.master.002.patch, HBASE-14997.master.003.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14997:
--
Attachment: HBASE-14997.master.003.patch
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch,
> HBASE-14997.master.002.patch, HBASE-14997.master.003.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17713) the interface '/version/cluster' with header 'Accept: application/json' return is not JSON but plain text
[
https://issues.apache.org/jira/browse/HBASE-17713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-17713:
--
Fix Version/s: 3.0.0
> the interface '/version/cluster' with header 'Accept: application/json'
> return is not JSON but plain text
> -
>
> Key: HBASE-17713
> URL: https://issues.apache.org/jira/browse/HBASE-17713
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 1.2.2
> Environment: Hhbase 1.2.2
>Reporter: Feng Ce
>Assignee: Guangxu Cheng
>Priority: Minor
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-17713.branch-1.patch, HBASE-17713.master.v1.patch,
> HBASE-17713.patch
>
> Original Estimate: 6h
> Remaining Estimate: 6h
>
> Hbase REST API, this interface `get 'version/cluster'`, when I use the header
> `Accept: application/json`, the response is not JSON but plain text.
> {code:none}
> curl -X GET \
> -H "Accept: application/json" \
> "http://localhost:/version/cluster";
>"1.2.2"
> {code}
> But when I use `Accept: text/xml`, the response is correct XML.
> {code:none}
> curl -X GET \
> -H "Accept: text/xml" \
> "http://localhost:/version/cluster";
> standalone="yes"?>1.2.2
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15931) Add log for long-running tasks in AsyncProcess
[
https://issues.apache.org/jira/browse/HBASE-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156552#comment-16156552
]
Lijian Shen commented on HBASE-15931:
-
Hi, @Yu Li, the patch has one defect on the LOG print.
LOG.info("#" + id + ", waiting for some tasks to finish. Expected
max=" + max
+ ", tasksInProgress=" + currentInProgress + " hasError=" +
hasError()
+ tableName == null ? "" : ", tableName=" + tableName);
Here need add one bracket for (tableName == null ? "" : ", tableName=" +
tableName), otherwise it will print wrong result because of operator priority,
current the print result is like, no other string logged:
", tableName=table1"
> Add log for long-running tasks in AsyncProcess
> --
>
> Key: HBASE-15931
> URL: https://issues.apache.org/jira/browse/HBASE-15931
> Project: HBase
> Issue Type: Improvement
> Components: Operability
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20, 1.1.6, 1.2.3
>
> Attachments: HBASE-15931.patch
>
>
> Currently if there's any long-tail tasks in a multi-action request like
> triggered by {{BufferedMutatorImpl#backgroundFlushCommits}}, we could see
> logging message like below
> {noformat}
> 2016-05-31 09:36:55,461 INFO [Thread-16]
> org.apache.hadoop.hbase.client.AsyncProcess: #28, waiting for some tasks to
> finish. Expected max=0, tasksInProgress=1
> {noformat}
> but there's no way to know detail of this long-tail, such as which
> table/region it's accessing. This JIRA aims at adding such log for easier
> debugging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156527#comment-16156527
]
Chia-Ping Tsai commented on HBASE-17732:
I can't access the link of design doc. Could you share the doc to us? Thanks.
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18765) The value of balancerRan is true even though no plans are executed
[
https://issues.apache.org/jira/browse/HBASE-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18765:
---
Description:
{code}
//We balance per group instead of per table
List plans = new ArrayList<>();
for(Map.Entry>> tableMap:
getRSGroupAssignmentsByTable(groupName).entrySet()) {
LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
+ tableMap.getValue());
List partialPlans =
balancer.balanceCluster(tableMap.getValue());
LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
partialPlans);
if (partialPlans != null) {
plans.addAll(partialPlans);
}
}
long startTime = System.currentTimeMillis();
balancerRan = plans != null;
{code}
The *plans* is never null.
was:
{code}
//We balance per group instead of per table
List plans = new ArrayList<>();
for(Map.Entry>> tableMap:
getRSGroupAssignmentsByTable(groupName).entrySet()) {
LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
+ tableMap.getValue());
List partialPlans =
balancer.balanceCluster(tableMap.getValue());
LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
partialPlans);
if (partialPlans != null) {
plans.addAll(partialPlans);
}
}
long startTime = System.currentTimeMillis();
balancerRan = plans != null;
{code}
The *plans* never be null.
> The value of balancerRan is true even though no plans are executed
> --
>
> Key: HBASE-18765
> URL: https://issues.apache.org/jira/browse/HBASE-18765
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18765.v0.patch
>
>
> {code}
> //We balance per group instead of per table
> List plans = new ArrayList<>();
> for(Map.Entry>> tableMap:
> getRSGroupAssignmentsByTable(groupName).entrySet()) {
> LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
> + tableMap.getValue());
> List partialPlans =
> balancer.balanceCluster(tableMap.getValue());
> LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
> partialPlans);
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> long startTime = System.currentTimeMillis();
> balancerRan = plans != null;
> {code}
> The *plans* is never null.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18763) quotas.TestQuotaThrottle failing on all nightly runs
[
https://issues.apache.org/jira/browse/HBASE-18763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156489#comment-16156489
]
Sean Busbey commented on HBASE-18763:
-
confirmed this is also in the branch-2 nightly runs
> quotas.TestQuotaThrottle failing on all nightly runs
>
>
> Key: HBASE-18763
> URL: https://issues.apache.org/jira/browse/HBASE-18763
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-3
>
>
> Recent flaky analysis has this test at 100% failure over 30 runs:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/10803/artifact/dashboard.html
> {code}
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserAndTableThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testNamespaceGlobalReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserGlobalThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserTableThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testTableGlobalThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testTableGlobalReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserGlobalReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserGlobalBypassThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserNamespaceThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testNamespaceGlobalThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserTableReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserNamespaceReadAndWriteThrottle
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18763) quotas.TestQuotaThrottle failing on all nightly runs
[
https://issues.apache.org/jira/browse/HBASE-18763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-18763:
Fix Version/s: 2.0.0-alpha-3
> quotas.TestQuotaThrottle failing on all nightly runs
>
>
> Key: HBASE-18763
> URL: https://issues.apache.org/jira/browse/HBASE-18763
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-3
>
>
> Recent flaky analysis has this test at 100% failure over 30 runs:
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/10803/artifact/dashboard.html
> {code}
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserAndTableThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testNamespaceGlobalReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserGlobalThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserTableThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testTableGlobalThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testTableGlobalReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserGlobalReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserGlobalBypassThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserNamespaceThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testNamespaceGlobalThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserTableReadAndWriteThrottle
> org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserNamespaceReadAndWriteThrottle
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156454#comment-16156454
]
stack commented on HBASE-14997:
---
Need a +1 here. This ok w/ folks? Thanks.
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch,
> HBASE-14997.master.002.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156448#comment-16156448 ] Anoop Sam John commented on HBASE-18298: Got some time to check Stack? Any suggestions? Once this is done, we have to similar cleanup for Region, Store etc. > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
[ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156446#comment-16156446 ] Anoop Sam John commented on HBASE-18375: Not yet pushed to 2.0? > The pool chunks from ChunkCreator are deallocated while in pool because there > is no reference to them > - > > Key: HBASE-18375 > URL: https://issues.apache.org/jira/browse/HBASE-18375 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-1 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18375-branch-2.patch, HBASE-18375-V01.patch, > HBASE-18375-V02-branch-2.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch, > HBASE-18375-V04.patch, HBASE-18375-V05.patch, HBASE-18375-V06.patch, > HBASE-18375-V07.patch, HBASE-18375-V08.patch, HBASE-18375-V09.patch, > HBASE-18375-V10.patch, HBASE-18375-V11.patch > > > Because MSLAB list of chunks was changed to list of chunk IDs, the chunks > returned back to pool can be deallocated by JVM because there is no reference > to them. The solution is to protect pool chunks from GC by the strong map of > ChunkCreator introduced by HBASE-18010. Will prepare the patch today. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156359#comment-16156359
]
Hadoop QA commented on HBASE-17732:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 40 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
21s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 24s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
18s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
40s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 52s{color}
| {color:red} hbase-endpoint in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
12s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m
29s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 47s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.access.TestAccessController |
| | hadoop.hbase.client.TestEnableTable |
| | hadoop.hbase.client.TestFromClientSide3 |
| | hadoop.hbase.client.TestServerBusyException |
| | hadoop.hbase.TestJMXConnectorServer |
| | hadoop.hbase.client.TestHCM |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.client.TestFromClientSide |
| | hadoop.hbase.regionserver.TestScannerWithCorruptHFile |
| | hadoop.hbase.c
[jira] [Commented] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156334#comment-16156334
]
Hadoop QA commented on HBASE-14997:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
18s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
24s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
11s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
40s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
27s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}192m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-14997 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885698/HBASE-14997.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5940f9f1c123 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
[jira] [Commented] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156325#comment-16156325 ] Hudson commented on HBASE-18768: FAILURE: Integrated in Jenkins build HBase-2.0 #469 (See [https://builds.apache.org/job/HBase-2.0/469/]) HBASE-18768 Move TestTableName to hbase-common from hbase-server (stack: rev 7592cb8d341ab747cb2fb1a2a12119d230b24557) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotClientRetries.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestScanEarlyTermination.java * (add) hbase-common/src/test/java/org/apache/hadoop/hbase/TestTableName.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerRetriableFailure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestWithDisabledAuthorization.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCorruptedRegionStoreFile.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18720) [C++] Update wangle dependency to pick up the new release with Apache License v2
[ https://issues.apache.org/jira/browse/HBASE-18720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-18720: > [C++] Update wangle dependency to pick up the new release with Apache License > v2 > > > Key: HBASE-18720 > URL: https://issues.apache.org/jira/browse/HBASE-18720 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Blocker > Fix For: HBASE-14850 > > Attachments: hbase-18720-v1.patch > > > Good news that Facebook decided to re-license the wangle library to use > Apache License v2. This unblocks the eventual merge of branch HBASE-14850 > since there will be no remaining licensing issues with dependencies. > However, we need to update the wangle dependency version to include the > commit > https://github.com/facebook/wangle/commit/5f686dee5753890f37326fcf38c396b7ba8981ee. > Next weekly release tag for wangle should be sufficient. > See > https://github.com/facebook/wangle/issues/105 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18720) [C++] Update wangle dependency to pick up the new release with Apache License v2
[
https://issues.apache.org/jira/browse/HBASE-18720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156326#comment-16156326
]
Ted Yu commented on HBASE-18720:
After upgrade, this command :
{code}
buck test --no-results-cache core:multi-retry-test
{code}
yields:
{code}
/usr/src/hbase/hbase-native-client/buck-out/gen/core/core#default,static/libcore.a(raw-async-table.cc.o):
In function `folly::Future > >::throwIfInvalid() const':
/usr/local/include/folly/futures/Future-inl.h:222: undefined reference to
`folly::throwNoState()'
/usr/local/include/folly/futures/Future-inl.h:222: undefined reference to
`folly::throwNoState()'
/usr/src/hbase/hbase-native-client/buck-out/gen/core/core#default,static/libcore.a(raw-async-table.cc.o):
In function `folly::Promise::throwIfFulfilled()':
/usr/local/include/folly/futures/Promise-inl.h:53: undefined reference to
`folly::throwNoState()'
/usr/local/include/folly/futures/Promise-inl.h:56: undefined reference to
`folly::throwPromiseAlreadySatisfied()'
/usr/src/hbase/hbase-native-client/buck-out/gen/core/core#default,static/libcore.a(raw-async-table.cc.o):
In function `folly::Try > >::throwIfFailed() const':
/usr/local/include/folly/Try-inl.h:134: undefined reference to
`folly::try_detail::throwUsingUninitializedTry()'
/usr/src/hbase/hbase-native-client/buck-out/gen/core/core#default,static/libcore.a(raw-async-table.cc.o):
In function `folly::Future > >::throwIfInvalid() const':
/usr/local/include/folly/futures/Future-inl.h:222: undefined reference to
`folly::throwNoState()'
/usr/local/include/folly/futures/Future-inl.h:222: undefined reference to
`folly::throwNoState()'
/usr/src/hbase/hbase-native-client/buck-out/gen/core/core#default,static/libcore.a(raw-async-table.cc.o):
In function `folly::Promise::throwIfFulfilled()':
/usr/local/include/folly/futures/Promise-inl.h:53: undefined reference to
`folly::throwNoState()'
/usr/local/include/folly/futures/Promise-inl.h:56: undefined reference to
`folly::throwPromiseAlreadySatisfied()'
{code}
> [C++] Update wangle dependency to pick up the new release with Apache License
> v2
>
>
> Key: HBASE-18720
> URL: https://issues.apache.org/jira/browse/HBASE-18720
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Blocker
> Fix For: HBASE-14850
>
> Attachments: hbase-18720-v1.patch
>
>
> Good news that Facebook decided to re-license the wangle library to use
> Apache License v2. This unblocks the eventual merge of branch HBASE-14850
> since there will be no remaining licensing issues with dependencies.
> However, we need to update the wangle dependency version to include the
> commit
> https://github.com/facebook/wangle/commit/5f686dee5753890f37326fcf38c396b7ba8981ee.
> Next weekly release tag for wangle should be sufficient.
> See
> https://github.com/facebook/wangle/issues/105
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18661) [HLC] Move clocks to a separate o.a.h.h.clocks package
[
https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156299#comment-16156299
]
Hadoop QA commented on HBASE-18661:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 24 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
53s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
13s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 28s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
27s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 71m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m
18s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay |
| | org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| | org.apache.hadoop.hbase.mapreduce.TestWALPlayer |
| | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplay |
| | org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestModifyTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.mapreduce.TestTableInputFormat |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
[jira] [Commented] (HBASE-18446) Mark StoreFileScanner as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156292#comment-16156292 ] Duo Zhang commented on HBASE-18446: --- If no objections I will provide a patch soon. We need to decide the final coprocessor API before beta release I think. Thanks. > Mark StoreFileScanner as IA.Private > --- > > Key: HBASE-18446 > URL: https://issues.apache.org/jira/browse/HBASE-18446 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > > Do not see any reason why it is marked as IA.LimitedPrivate. It is not > referenced in any CPs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18444) Add support for specifying custom meta table suffix
[
https://issues.apache.org/jira/browse/HBASE-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reopened HBASE-18444:
> Add support for specifying custom meta table suffix
> ---
>
> Key: HBASE-18444
> URL: https://issues.apache.org/jira/browse/HBASE-18444
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Labels: patch
> Attachments: HBASE-18444.branch-1.001.patch,
> HBASE-18444.master.001.patch
>
>
> This patch provides user an ability to have a custom meta table suffix.
> In order to share a common rootDir amongst multiple HBase clusters, we need
> separate meta table for each cluster.
> With custom meta table suffix, the meta table is named in the form of
> meta_{suffix} making it specific to a cluster.
> Users can specify the meta table suffix as part of hbase-site.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18444) Add support for specifying custom meta table suffix
[
https://issues.apache.org/jira/browse/HBASE-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156287#comment-16156287
]
Ted Yu commented on HBASE-18444:
Reverted previous patch and applied HBASE-18768 to HBASE-18477 branch.
Ajay:
Can you attach new patch.
> Add support for specifying custom meta table suffix
> ---
>
> Key: HBASE-18444
> URL: https://issues.apache.org/jira/browse/HBASE-18444
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Labels: patch
> Attachments: HBASE-18444.branch-1.001.patch,
> HBASE-18444.master.001.patch
>
>
> This patch provides user an ability to have a custom meta table suffix.
> In order to share a common rootDir amongst multiple HBase clusters, we need
> separate meta table for each cluster.
> With custom meta table suffix, the meta table is named in the form of
> meta_{suffix} making it specific to a cluster.
> Users can specify the meta table suffix as part of hbase-site.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156285#comment-16156285
]
xinxin fan commented on HBASE-16894:
[#Mikhail Antonov ]
I have review your patch and address the feedback from [#Ted yu] and [#Yi
Liang]. yes, i found the same problem that you faced, let me dig into it.
{quote}And I checked your code in HBASE-18090, your changes mainly go to
TableSnapshotInputFormat, and this one is to change TableInputFormatBase. They
are using same same idea, but we still need two jiras to handle them, they are
not a duplicates. And I will first review your code.{quote}
let me work on the jira HBASE-18090? [#Mikhail Antonov ]
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Attachments: ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156282#comment-16156282 ] Hudson commented on HBASE-18768: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3671 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3671/]) HBASE-18768 Move TestTableName to hbase-common from hbase-server (stack: rev 3a9dc8fbd58ab6dc4a895cbcf9644422fb5b8223) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCorruptedRegionStoreFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotClientRetries.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestWithDisabledAuthorization.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerRetriableFailure.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java * (add) hbase-common/src/test/java/org/apache/hadoop/hbase/TestTableName.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestScanEarlyTermination.java > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18768: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0-alpha-3 Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks for review Ajay... > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18610) Provide capability to activate chaos monkey
[ https://issues.apache.org/jira/browse/HBASE-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18610: --- Attachment: 18610.v8.txt Patch v8 removes the code from MonkeyRunner destructor. > Provide capability to activate chaos monkey > --- > > Key: HBASE-18610 > URL: https://issues.apache.org/jira/browse/HBASE-18610 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu > Attachments: 18610.v1.txt, 18610.v4.txt, 18610.v5.txt, 18610.v6.txt, > 18610.v7.txt, 18610.v8.txt > > > Currently load-client runs against a cluster where region servers are stable. > We need to introduce chaos monkey so that wider coverage for read path is > exercised. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18610) Provide capability to activate chaos monkey
[ https://issues.apache.org/jira/browse/HBASE-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156274#comment-16156274 ] Ted Yu edited comment on HBASE-18610 at 9/7/17 1:08 AM: bq. ChaosMonkeyRunner must run againt a distributed cluster As I mentioned on Aug 24th, you should pass conf directory path containing the hbase-site.xml for remote cluster to load-client which would pass ../conf/hbase-site.xml (from your example) to monkey runner. was (Author: yuzhih...@gmail.com): bq. ChaosMonkeyRunner must run againt a distributed cluster As I mentioned on Aug 24th, you should pass conf directory path to load-client which would pass ../conf/hbase-site.xml (from your example) to monkey runner. > Provide capability to activate chaos monkey > --- > > Key: HBASE-18610 > URL: https://issues.apache.org/jira/browse/HBASE-18610 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu > Attachments: 18610.v1.txt, 18610.v4.txt, 18610.v5.txt, 18610.v6.txt, > 18610.v7.txt > > > Currently load-client runs against a cluster where region servers are stable. > We need to introduce chaos monkey so that wider coverage for read path is > exercised. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18610) Provide capability to activate chaos monkey
[ https://issues.apache.org/jira/browse/HBASE-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156274#comment-16156274 ] Ted Yu commented on HBASE-18610: bq. ChaosMonkeyRunner must run againt a distributed cluster As I mentioned on Aug 24th, you should pass conf directory path to load-client which would pass ../conf/hbase-site.xml (from your example) to monkey runner. > Provide capability to activate chaos monkey > --- > > Key: HBASE-18610 > URL: https://issues.apache.org/jira/browse/HBASE-18610 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu > Attachments: 18610.v1.txt, 18610.v4.txt, 18610.v5.txt, 18610.v6.txt, > 18610.v7.txt > > > Currently load-client runs against a cluster where region servers are stable. > We need to introduce chaos monkey so that wider coverage for read path is > exercised. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18610) Provide capability to activate chaos monkey
[
https://issues.apache.org/jira/browse/HBASE-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156265#comment-16156265
]
Enis Soztutar commented on HBASE-18610:
---
[~ted_yu] I was testing this, but again could not get it working. Two problems
when I run these in the docker:
If CM is not launched, the job failed at the end:
{code}
root@securecluster:/usr/src/hbase/hbase-native-client#
./buck-out/gen/core/load-client -num_rows=10 -appends=false
-report_num_rows=1000 -num_cols=10 -conf ../conf/
W0907 00:30:47.859833 8590 hbase-configuration-loader.cc:123] Unable to open
file[../conf/hbase-default.xml]
I0907 00:30:47.864672 8590 location-cache.cc:77] Connecting to ZooKeeper.
Quorum:localhost:2181
2017-09-07 00:30:47,864:8590(0x7fc36a141840):ZOO_INFO@log_env@726: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2017-09-07 00:30:47,864:8590(0x7fc36a141840):ZOO_INFO@log_env@730: Client
environment:host.name=securecluster
2017-09-07 00:30:47,864:8590(0x7fc36a141840):ZOO_INFO@log_env@737: Client
environment:os.name=Linux
2017-09-07 00:30:47,864:8590(0x7fc36a141840):ZOO_INFO@log_env@738: Client
environment:os.arch=3.10.0-514.2.2.el7.x86_64
2017-09-07 00:30:47,864:8590(0x7fc36a141840):ZOO_INFO@log_env@739: Client
environment:os.version=#1 SMP Tue Dec 6 23:06:41 UTC 2016
2017-09-07 00:30:47,865:8590(0x7fc36a141840):ZOO_INFO@log_env@747: Client
environment:user.name=(null)
2017-09-07 00:30:47,865:8590(0x7fc36a141840):ZOO_INFO@log_env@755: Client
environment:user.home=/root
2017-09-07 00:30:47,865:8590(0x7fc36a141840):ZOO_INFO@log_env@767: Client
environment:user.dir=/usr/src/hbase/hbase-native-client
2017-09-07 00:30:47,865:8590(0x7fc36a141840):ZOO_INFO@zookeeper_init@800:
Initiating client connection, host=localhost:2181 sessionTimeout=9
watcher=(nil) sessionId=0 sessionPasswd= context=(nil) flags=0
I0907 00:30:47.866176 8590 load-client.cc:330] Sending put requests
2017-09-07 00:30:47,872:8590(0x7fc3555e8700):ZOO_INFO@check_events@1728:
initiated connection to server [::1:2181]
2017-09-07 00:30:47,877:8590(0x7fc3555e8700):ZOO_INFO@check_events@1775:
session establishment complete on server [::1:2181],
sessionId=0x15e59be46e30006, negotiated timeout=9
I0907 00:30:51.234663 8615 load-client.cc:231] (Thread 9) Written 1000 rows in
3361 ms.
I0907 00:30:51.250079 8608 load-client.cc:231] (Thread 2) Written 1000 rows in
3381 ms.
I0907 00:30:51.254568 8613 load-client.cc:231] (Thread 7) Written 1000 rows in
3381 ms.
I0907 00:30:51.291641 8610 load-client.cc:231] (Thread 4) Written 1000 rows in
3420 ms.
I0907 00:31:17.682425 8630 load-client.cc:209] (Thread 1) Sent 1 gets in
2886 ms.
I0907 00:31:17.683099 8590 load-client.cc:396] Successful. sent multi-get
requests for 10 rows in 2888 ms.
*** Aborted at 1504744277 (unix time) try "date -d @1504744277" if you are
using GNU date ***
PC: @ 0x422e85 hbase::MonkeyRunner::~MonkeyRunner()
*** SIGSEGV (@0x0) received by PID 8590 (TID 0x7fc36a141840) from PID 0; stack
trace: ***
@ 0x7fc3681033d0 (unknown)
@ 0x422e85 hbase::MonkeyRunner::~MonkeyRunner()
@ 0x41fb04 main
@ 0x7fc366fc3830 __libc_start_main
@ 0x418e29 _start
@0x0 (unknown)
Segmentation fault (core dumped)
{code}
If I launch it with the CM tool, then it fails with:
{code}
root@securecluster:/usr/src/hbase/hbase-native-client#
./buck-out/gen/core/load-client -num_rows=10 -appends=false
-report_num_rows=1000 -num_cols=10 -conf /usr/src/hbase/conf/ -monkey
serverKilling
W0907 00:42:58.259927 8838 hbase-configuration-loader.cc:123] Unable to open
file[/usr/src/hbase/conf/hbase-default.xml]
I0907 00:42:58.260288 8839 mini-cluster.cc:42] got empty classpath
I0907 00:42:58.260357 8839 mini-cluster.cc:51] set clspath 0
I0907 00:42:58.271355 8838 location-cache.cc:77] Connecting to ZooKeeper.
Quorum:localhost:2181
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@726: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@730: Client
environment:host.name=securecluster
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@737: Client
environment:os.name=Linux
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@738: Client
environment:os.arch=3.10.0-514.2.2.el7.x86_64
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@739: Client
environment:os.version=#1 SMP Tue Dec 6 23:06:41 UTC 2016
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@747: Client
environment:user.name=(null)
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@755: Client
environment:user.home=/root
2017-09-07 00:42:58,271:8838(0x7fcc53f86840):ZOO_INFO@log_env@767: Client
environment:user.dir=/usr/src/hbase/hbase-native-client
2017-09-07 00:42:58,271:88
[jira] [Updated] (HBASE-18541) [C++] Segfaults from JNI
[
https://issues.apache.org/jira/browse/HBASE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18541:
---
Attachment: 18541.v3.txt
Tried patch v3.
Without AttachCurrentThread() / DetachCurrentThread() (i.e. lock only), I got
crash similar to the current.
With AttachCurrentThread() / DetachCurrentThread(), I got the following:
{code}
*** SIGSEGV (@0x258) received by PID 160 (TID 0x7f8eaa021840) from PID 600;
stack trace: ***
@ 0x7f8ea97954e2 (unknown)
@ 0x7f8ea9799939 JVM_handle_linux_signal
@ 0x7f8ea978d838 (unknown)
@ 0x7f8ea7fe33d0 (unknown)
@ 0x7f8ea9550703 (unknown)
@ 0x7f8ea9552db6 (unknown)
@ 0x43f2f3 hbase::MiniCluster::GetConfValue()
@ 0x439aaa hbase::TestUtil::StartMiniCluster()
@ 0x59b7de
testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x57cbbf testing::TestCase::Run()
{code}
> [C++] Segfaults from JNI
>
>
> Key: HBASE-18541
> URL: https://issues.apache.org/jira/browse/HBASE-18541
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Ted Yu
> Attachments: 18541.v1.txt, 18541.v3.txt
>
>
> retry-test and multi-retry-test fails flakily when run with
> {code}
> buck test --all --no-results-cache
> {code}
> or when run in a loop:
> {code}
> for i in `seq 1 10`; do buck test --no-results-cache core:retry-test || break
> 1; done
> {code}
> The problem seems to be within the JNI internals and usually happens at the
> create table method call. I was not able to inspect much, but the comments in
> our mini-cluster indicate that we may need to use global references instead
> of local ones. I suspect the problem happens when there is a GC run for the
> test since the failure happens usually after some time (but almost always in
> create table method).
> [~ted_yu] do you mind taking a look at this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17732:
-
Attachment: HBASE-17732.master.004.patch
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18720) [C++] Update wangle dependency to pick up the new release with Apache License v2
[
https://issues.apache.org/jira/browse/HBASE-18720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156220#comment-16156220
]
Ted Yu commented on HBASE-18720:
{code}
make[4]: Entering directory '/usr/src/folly-2017.09.04.00/folly/test'
PASS: sorted_vector_types_test
PASS: foreach_test
PASS: hash_test
PASS: timeout_queue_test
PASS: conv_test
PASS: expected_test
PASS: range_test
PASS: math_test
PASS: bits_test
PASS: bit_iterator_test
PASS: spin_lock_test
PASS: array_test
PASS: constexpr_math_test
PASS: small_locks_test
PASS: packed_sync_ptr_test
{code}
Do we have to run these tests ?
> [C++] Update wangle dependency to pick up the new release with Apache License
> v2
>
>
> Key: HBASE-18720
> URL: https://issues.apache.org/jira/browse/HBASE-18720
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Blocker
> Fix For: HBASE-14850
>
> Attachments: hbase-18720-v1.patch
>
>
> Good news that Facebook decided to re-license the wangle library to use
> Apache License v2. This unblocks the eventual merge of branch HBASE-14850
> since there will be no remaining licensing issues with dependencies.
> However, we need to update the wangle dependency version to include the
> commit
> https://github.com/facebook/wangle/commit/5f686dee5753890f37326fcf38c396b7ba8981ee.
> Next weekly release tag for wangle should be sufficient.
> See
> https://github.com/facebook/wangle/issues/105
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17732:
-
Description:
The two main changes are:
* *Adding template for coprocessor type to CoprocessorEnvironment i.e.
{{interface CoprocessorEnvironment}}*
** Enables us to load only relevant coprocessors in hosts. Right now each
type of host loads all types of coprocs and it's only during execOperation that
it checks if the coproc is of correct type i.e. XCoprocessorHost will load
XObserver, YObserver, and all others, and will check in execOperation if
{{coproc instanceOf XObserver}} and ignore the rest.
** Allow sharing of a bunch functions/classes which are currently duplicated
in each host. For eg. CoprocessorOperations, CoprocessorOperationWithResult,
execOperations().
* *Introduce 4 coprocessor classes and use composition between these new
classes and and old observers*
** The real gold here is, moving forward, we'll be able to break down giant
everything-in-one observers (masterobserver has 100+ functions) into smaller,
more focused observers. These smaller observer can then have different compat
guarantees!!
Here's a more detailed design doc:
https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
was:
Adding template for coprocessor type to CoprocessorEnvironment will:
- allow us to load only relevant coprocessors in hosts (right now each type of
host loads all types of coprocs and it's only during execOperation that it
checks if the coproc is of correct type i.e. XCoprocessorHost will load
XObserver, YObserver, and all others, and will check in execOperation if
{{coproc instanceOf XObserver}} and ignore the rest).
- allow sharing of a bunch functions/classes which are currently 99% duplicate
of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult,
execOperations().
Here's a more detailed design doc:
https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156206#comment-16156206 ] Ajay Jadhav commented on HBASE-18768: - +1 lgtm > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-17732:
-
Description:
Adding template for coprocessor type to CoprocessorEnvironment will:
- allow us to load only relevant coprocessors in hosts (right now each type of
host loads all types of coprocs and it's only during execOperation that it
checks if the coproc is of correct type i.e. XCoprocessorHost will load
XObserver, YObserver, and all others, and will check in execOperation if
{{coproc instanceOf XObserver}} and ignore the rest).
- allow sharing of a bunch functions/classes which are currently 99% duplicate
of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult,
execOperations().
Here's a more detailed design doc:
https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
was:
Adding template for coprocessor type to CoprocessorEnvironment will:
- get rid of 'instanceOf' checks we do in various coprocessor hosts
- allow us to load only relevant coprocessors in hosts (right now i think each
host loads all coprocs and it's only during execOperation that it checks if the
coproc is of correct type ie XXCoprocessorHost will load XXObserver,
YYObserver, and all others, and in will check in execOperation if coproc is of
type XXObserver and ignore the rest).
- allow sharing of a bunch functions/classes which are currently 99% duplicate
of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult,
execOperations().
Here's a more detailed design doc:
https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch
>
>
> Adding template for coprocessor type to CoprocessorEnvironment will:
> - allow us to load only relevant coprocessors in hosts (right now each type
> of host loads all types of coprocs and it's only during execOperation that it
> checks if the coproc is of correct type i.e. XCoprocessorHost will load
> XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest).
> - allow sharing of a bunch functions/classes which are currently 99%
> duplicate of each other. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-rOpQmZuixCDua-LT9X4/edit
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[
https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156198#comment-16156198
]
Hadoop QA commented on HBASE-18768:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 8s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18768 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885666/HBASE-18768.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 01a23ac12209 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / fd49a98 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8482/testReport/ |
| modules | C: hbase-common hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8482/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Move TestT
[jira] [Resolved] (HBASE-18720) [C++] Update wangle dependency to pick up the new release with AL v2
[ https://issues.apache.org/jira/browse/HBASE-18720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-18720. --- Resolution: Fixed Assignee: Enis Soztutar > [C++] Update wangle dependency to pick up the new release with AL v2 > > > Key: HBASE-18720 > URL: https://issues.apache.org/jira/browse/HBASE-18720 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Blocker > Fix For: HBASE-14850 > > Attachments: hbase-18720-v1.patch > > > Good news that Facebook decided to re-license the wangle library to use > Apache License v2. This unblocks the eventual merge of branch HBASE-14850 > since there will be no remaining licensing issues with dependencies. > However, we need to update the wangle dependency version to include the > commit > https://github.com/facebook/wangle/commit/5f686dee5753890f37326fcf38c396b7ba8981ee. > Next weekly release tag for wangle should be sufficient. > See > https://github.com/facebook/wangle/issues/105 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18720) [C++] Update wangle dependency to pick up the new release with Apache License v2
[ https://issues.apache.org/jira/browse/HBASE-18720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18720: -- Summary: [C++] Update wangle dependency to pick up the new release with Apache License v2 (was: [C++] Update wangle dependency to pick up the new release with AL v2) > [C++] Update wangle dependency to pick up the new release with Apache License > v2 > > > Key: HBASE-18720 > URL: https://issues.apache.org/jira/browse/HBASE-18720 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Blocker > Fix For: HBASE-14850 > > Attachments: hbase-18720-v1.patch > > > Good news that Facebook decided to re-license the wangle library to use > Apache License v2. This unblocks the eventual merge of branch HBASE-14850 > since there will be no remaining licensing issues with dependencies. > However, we need to update the wangle dependency version to include the > commit > https://github.com/facebook/wangle/commit/5f686dee5753890f37326fcf38c396b7ba8981ee. > Next weekly release tag for wangle should be sufficient. > See > https://github.com/facebook/wangle/issues/105 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18720) [C++] Update wangle dependency to pick up the new release with AL v2
[
https://issues.apache.org/jira/browse/HBASE-18720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-18720:
--
Attachment: hbase-18720-v1.patch
Wangle version needs to be at least {{2017.09.04.00}}, which depends on a
recent version of folly as well. Both can be upgraded to {{v2017.09.04.00}}.
I've tested this patch with docker/buck and Makefile as well. Will commit
shortly.
> [C++] Update wangle dependency to pick up the new release with AL v2
>
>
> Key: HBASE-18720
> URL: https://issues.apache.org/jira/browse/HBASE-18720
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Priority: Blocker
> Fix For: HBASE-14850
>
> Attachments: hbase-18720-v1.patch
>
>
> Good news that Facebook decided to re-license the wangle library to use
> Apache License v2. This unblocks the eventual merge of branch HBASE-14850
> since there will be no remaining licensing issues with dependencies.
> However, we need to update the wangle dependency version to include the
> commit
> https://github.com/facebook/wangle/commit/5f686dee5753890f37326fcf38c396b7ba8981ee.
> Next weekly release tag for wangle should be sufficient.
> See
> https://github.com/facebook/wangle/issues/105
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18727) [C++] Don't link JNI/JVM stuff into libHbaseClient
[ https://issues.apache.org/jira/browse/HBASE-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156179#comment-16156179 ] Enis Soztutar commented on HBASE-18727: --- [~sudeeps] can you please take a look at this patch to see it makes sense or not? > [C++] Don't link JNI/JVM stuff into libHbaseClient > -- > > Key: HBASE-18727 > URL: https://issues.apache.org/jira/browse/HBASE-18727 > Project: HBase > Issue Type: Sub-task >Reporter: Scott Hunt >Priority: Minor > Fix For: HBASE-14850 > > Attachments: hbase-18727-v2.patch, Makefile_HBASE_18727.patch > > > It doesn't seem like we should be linking the objects from the test-util > directory into the library, as those [I believe] are just for testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18661) [HLC] Move clocks to a separate o.a.h.h.clocks package
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18661: - Attachment: HBASE-18661.HBASE-14070.HLC.002.patch > [HLC] Move clocks to a separate o.a.h.h.clocks package > -- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18661.HBASE-14070.HLC.001.patch, > HBASE-18661.HBASE-14070.HLC.002.patch > > > --I think we can move all clocks related code out of h-common and h-server > into a new module which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency > structure.-- > Edit: let's just move to a separate package in hbase-common for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14997:
--
Attachment: HBASE-14997.master.002.patch
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch,
> HBASE-14997.master.002.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14997:
--
Assignee: stack
Status: Patch Available (was: Open)
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156172#comment-16156172
]
stack commented on HBASE-14997:
---
.001 swap in a generic ComparatorOperator class in place of the filter
CompareOps. Plumb it all the ways through. Fixes API.
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14997:
--
Attachment: HBASE-14997.master.001.patch
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-14997.master.001.patch
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18579) Enable core dump by default for docker
[ https://issues.apache.org/jira/browse/HBASE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156144#comment-16156144 ] Ted Yu commented on HBASE-18579: [~enis]: Can you take a look at patch v2 ? > Enable core dump by default for docker > -- > > Key: HBASE-18579 > URL: https://issues.apache.org/jira/browse/HBASE-18579 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Minor > Fix For: HBASE-14850 > > Attachments: 18579.v1.txt, 18579.v2.txt > > > In recent debugging experience, I found that by default the ulimit value > prohibits the generation of core dump. This makes debugging difficult. > We should enable core dump generation by default. > https://www.akadia.com/services/ora_enable_core.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156133#comment-16156133 ] Hudson commented on HBASE-18652: FAILURE: Integrated in Jenkins build HBase-1.4 #906 (See [https://builds.apache.org/job/HBase-1.4/906/]) HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (tedyu: rev 1cca36b3b57067b6f550792433f28427f9f31fcd) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18737) Display configured max size of memstore and cache on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156093#comment-16156093
]
Hudson commented on HBASE-18737:
FAILURE: Integrated in Jenkins build HBase-1.4 #905 (See
[https://builds.apache.org/job/HBase-1.4/905/])
HBASE-18737 Display configured max size of cache on RS UI (tedyu: rev
bc3c7292767392507e3ff2b5e3f0514ba0426436)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java
* (edit)
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* (edit)
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon
* (edit)
hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java
* (edit)
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
> Display configured max size of memstore and cache on RS UI
> --
>
> Key: HBASE-18737
> URL: https://issues.apache.org/jira/browse/HBASE-18737
> Project: HBase
> Issue Type: Improvement
>Reporter: Biju Nair
>Assignee: Biju Nair
>Priority: Minor
> Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18737-BRANCH-1.PATCH, HBASE-18737.PATCH,
> HBASE-18737-WIP.PATCH, RS_UI_AFTER-1.PNG, RS_UI_AFTER.PNG, RS_UI_FULL.PNG,
> RS_UI_L1.PNG, RS_UI_L2.PNG, RS_UI.PNG
>
>
> Displaying the configured size of {{memstore}} and {{cache}} will help
> non-admin users understand the cluster capacity. Attached screenshot with
> proposed usability related changes and the current RS UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156037#comment-16156037 ] Hudson commented on HBASE-18652: FAILURE: Integrated in Jenkins build HBase-1.5 #51 (See [https://builds.apache.org/job/HBase-1.5/51/]) HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (tedyu: rev 5bfe1da9847b88e1ee81bc5e166faf493f042d2d) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14997) Move compareOp and Comparators out of filter to client package
[
https://issues.apache.org/jira/browse/HBASE-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156024#comment-16156024
]
stack commented on HBASE-14997:
---
DynamoDB has notion of a Condition and ComparisonOperator.
http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Condition.html
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/dynamodbv2/model/ComparisonOperator.html
You stipulate KeyCondition like this (in JSON) as part of a Query
This late in the game, let me just fix the CompareOp reveal in API. Condition
would be nice... say a CellCondition... but would have to teach users new means
of using API.
> Move compareOp and Comparators out of filter to client package
> --
>
> Key: HBASE-14997
> URL: https://issues.apache.org/jira/browse/HBASE-14997
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
>
> {{Table.checkAndPut()}} and its cousins depend on CompareOp from the filter
> package. Originally, ComparaOp and ByteArrayComparable, and various
> "comparators" have been used in filters, so these are in the filter package.
> However, for checkAndPut(), etc we depend on the filter subpackage although
> these are not filter related operations. We can use some clean up.
> {code}
> boolean checkAndPut(byte[] row, byte[] family, byte[] qualifier,
> CompareFilter.CompareOp compareOp, byte[] value, Put put) throws
> IOException;
> {code}
> Some ideas
> - Cleanup ByteArrayComparable interface (see the TODO at the class)
> - Maybe introduce a {{Condition}} or a similar concept and do
> {{checkAndPut(Condition condition, Put put)}} and change filters to use that
> as well.
> - Introducing Condition like thing will allow us to have an interface like:
> {{checkAndMutate(List conditions, List mutations)}}.
> - BinaryComparator, etc are not "Comparators", they are comparables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156016#comment-16156016 ] stack commented on HBASE-17980: --- bq. I didn't change Admin Interface API in this patch, should I replace all HRegionInfo to RegionInfo in this patch ??? Yes. Admin is going to be incompatible in hbase2. It is 'ok'. I see three methods that need to be changed. RegionReplicaUtil is audience private so ok to change. Yeah, looks like getMetaTableRowResultAsSplitRegion is never used. Try purging it? Kill MutableRegionInfo I'd say. You've done this lovely cleanup. MutableRegionInfo hanging around makes it a little dirty (smile). > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0 > > Attachments: HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156011#comment-16156011 ] Biju Nair commented on HBASE-18652: --- Thanks [~tedyu]. Will take care of the patch file name in the future. > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18444) Add support for specifying custom meta table suffix
[
https://issues.apache.org/jira/browse/HBASE-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155996#comment-16155996
]
Ajay Jadhav commented on HBASE-18444:
-
Thanks [~stack], [~yuzhih...@gmail.com] and [~ashish singhi] for the reviews.
I'll keep an eye on HBASE-18768 and update the attached patch.
> Add support for specifying custom meta table suffix
> ---
>
> Key: HBASE-18444
> URL: https://issues.apache.org/jira/browse/HBASE-18444
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Labels: patch
> Attachments: HBASE-18444.branch-1.001.patch,
> HBASE-18444.master.001.patch
>
>
> This patch provides user an ability to have a custom meta table suffix.
> In order to share a common rootDir amongst multiple HBase clusters, we need
> separate meta table for each cluster.
> With custom meta table suffix, the meta table is named in the form of
> meta_{suffix} making it specific to a cluster.
> Users can specify the meta table suffix as part of hbase-site.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18652: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0-alpha-3 1.5.0 1.4.0 Status: Resolved (was: Patch Available) Thanks for the patch, Biju. > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[
https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155990#comment-16155990
]
Hadoop QA commented on HBASE-18652:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18652 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18652 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885663/HBASE-18652-BRANCH-1.PATCH
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8483/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Expose individual cache stats in a CombinedCache through JMX
>
>
> Key: HBASE-18652
> URL: https://issues.apache.org/jira/browse/HBASE-18652
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Biju Nair
>Assignee: Biju Nair
> Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH,
> HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH
>
>
> With offHeap cache being used to store data blocks and on-heap for index and
> bloom filters, exposing the stats from the individual caches through JMX will
> help understand the cache usage trend. Currently the combined cache stats is
> available through JMX which may not provide insight into the individual cache
> usage.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155988#comment-16155988 ] Ted Yu commented on HBASE-18652: In the future, name the 'BRANCH-1' as branch-1 : QA only recognizes branch in lower case. > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18444) Add support for specifying custom meta table suffix
[
https://issues.apache.org/jira/browse/HBASE-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155982#comment-16155982
]
stack commented on HBASE-18444:
---
I filed HBASE-18768. Let me make sure it good w/ hadoopqa. Then you can make
stuff private as addendum here. Thanks [~ajayjadhav]
> Add support for specifying custom meta table suffix
> ---
>
> Key: HBASE-18444
> URL: https://issues.apache.org/jira/browse/HBASE-18444
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Labels: patch
> Attachments: HBASE-18444.branch-1.001.patch,
> HBASE-18444.master.001.patch
>
>
> This patch provides user an ability to have a custom meta table suffix.
> In order to share a common rootDir amongst multiple HBase clusters, we need
> separate meta table for each cluster.
> With custom meta table suffix, the meta table is named in the form of
> meta_{suffix} making it specific to a cluster.
> Users can specify the meta table suffix as part of hbase-site.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18768: -- Assignee: stack Status: Patch Available (was: Open) > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
[ https://issues.apache.org/jira/browse/HBASE-18768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18768: -- Attachment: HBASE-18768.master.001.patch > Move TestTableName to hbase-common from hbase-server > > > Key: HBASE-18768 > URL: https://issues.apache.org/jira/browse/HBASE-18768 > Project: HBase > Issue Type: Task >Reporter: stack > Attachments: HBASE-18768.master.001.patch > > > HBASE-18444 notices that TableName test is in a different module altogether > (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated HBASE-18652: -- Attachment: HBASE-18652-BRANCH-1.PATCH Attaching patch for branch 1. > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: HBASE-18652-BRANCH-1.PATCH, HBASE-18652.PATCH, > HBASE-18652-V1.0.PATCH, HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18768) Move TestTableName to hbase-common from hbase-server
stack created HBASE-18768: - Summary: Move TestTableName to hbase-common from hbase-server Key: HBASE-18768 URL: https://issues.apache.org/jira/browse/HBASE-18768 Project: HBase Issue Type: Task Reporter: stack HBASE-18444 notices that TableName test is in a different module altogether (hbase-server). Fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15607) Remove PB references from Admin for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155960#comment-16155960 ] stack commented on HBASE-15607: --- GO for it [~ram_krish] > Remove PB references from Admin for 2.0 > --- > > Key: HBASE-15607 > URL: https://issues.apache.org/jira/browse/HBASE-15607 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0, 1.3.0, 2.0.0-alpha-3 > > Attachments: HBASE-15607_1.patch, HBASE-15607_2.patch, > HBASE-15607_3.patch, HBASE-15607_3.patch, HBASE-15607_4.patch, > HBASE-15607_4.patch, HBASE-15607_branch-1.3.patch, > HBASE-15607_branch-1.patch, HBASE-15607.patch > > > This is a sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155959#comment-16155959
]
Yi Liang commented on HBASE-16894:
--
{quote}oh wait, you mean difference is that here it's about live tables, not
snapshots?{quote}
yes, this one is for live tables
I am now testing my code, i will see if I face same problem as yours.
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Attachments: ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18444) Add support for specifying custom meta table suffix
[
https://issues.apache.org/jira/browse/HBASE-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155957#comment-16155957
]
Ajay Jadhav commented on HBASE-18444:
-
[~stack] The reason for keeping those methods public was because the
TestTableName.java is in hbase-server module (and TableName.java in
hbase-common) where I'm using
these methods for unit testing.
I'm not sure why the TestTableName.java should be in hbase-server module as it
is entirely testing the TableName class. Should I move this to hbase-common?
For HBaseConfiguration.create(): Yes, since the META_TABLE_NAME is static, it
will be initialized only during startup once.
> Add support for specifying custom meta table suffix
> ---
>
> Key: HBASE-18444
> URL: https://issues.apache.org/jira/browse/HBASE-18444
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Labels: patch
> Attachments: HBASE-18444.branch-1.001.patch,
> HBASE-18444.master.001.patch
>
>
> This patch provides user an ability to have a custom meta table suffix.
> In order to share a common rootDir amongst multiple HBase clusters, we need
> separate meta table for each cluster.
> With custom meta table suffix, the meta table is named in the form of
> meta_{suffix} making it specific to a cluster.
> Users can specify the meta table suffix as part of hbase-site.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18737) Display configured max size of memstore and cache on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155955#comment-16155955
]
Hudson commented on HBASE-18737:
FAILURE: Integrated in Jenkins build HBase-1.5 #50 (See
[https://builds.apache.org/job/HBase-1.5/50/])
HBASE-18737 Display configured max size of cache on RS UI (tedyu: rev
5de4a7d898f6a6f9be0e7759504ca165b4a726e0)
* (edit)
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon
* (edit)
hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java
* (edit)
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java
* (edit)
hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon
> Display configured max size of memstore and cache on RS UI
> --
>
> Key: HBASE-18737
> URL: https://issues.apache.org/jira/browse/HBASE-18737
> Project: HBase
> Issue Type: Improvement
>Reporter: Biju Nair
>Assignee: Biju Nair
>Priority: Minor
> Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18737-BRANCH-1.PATCH, HBASE-18737.PATCH,
> HBASE-18737-WIP.PATCH, RS_UI_AFTER-1.PNG, RS_UI_AFTER.PNG, RS_UI_FULL.PNG,
> RS_UI_L1.PNG, RS_UI_L2.PNG, RS_UI.PNG
>
>
> Displaying the configured size of {{memstore}} and {{cache}} will help
> non-admin users understand the cluster capacity. Attached screenshot with
> proposed usability related changes and the current RS UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18697) Need a shaded hbase-mapreduce module
[ https://issues.apache.org/jira/browse/HBASE-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155954#comment-16155954 ] Hudson commented on HBASE-18697: FAILURE: Integrated in Jenkins build HBase-2.0 #467 (See [https://builds.apache.org/job/HBase-2.0/467/]) HBASE-18697 Replace hbase-shaded-server jar with hbase-shaded-mapreduce (appy: rev e25401db493b523d654d27ce333ec9f32a13e417) * (add) hbase-shaded/hbase-shaded-mapreduce/pom.xml * (edit) hbase-shaded/hbase-shaded-check-invariants/pom.xml * (edit) hbase-shaded/pom.xml * (delete) hbase-shaded/hbase-shaded-server/pom.xml > Need a shaded hbase-mapreduce module > > > Key: HBASE-18697 > URL: https://issues.apache.org/jira/browse/HBASE-18697 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Duo Zhang >Assignee: Appy > Fix For: 3.0.0, 2.0.0-alpha-3 > > Attachments: HBASE-18697.master.001.patch, > HBASE-18697.master.002.patch, HBASE-18697.master.003.patch, > HBASE-18697.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17713) the interface '/version/cluster' with header 'Accept: application/json' return is not JSON but plain text
[
https://issues.apache.org/jira/browse/HBASE-17713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155952#comment-16155952
]
Hudson commented on HBASE-17713:
FAILURE: Integrated in Jenkins build HBase-2.0 #467 (See
[https://builds.apache.org/job/HBase-2.0/467/])
HBASE-17713 the interface '/version/cluster' with header 'Accept: (tedyu: rev
d7a74a75a133810ec5159ede1e451b2e0f281310)
* (edit)
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/client/TestXmlParsing.java
* (edit)
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestVersionResource.java
* (edit)
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/StorageClusterVersionModel.java
* (edit)
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestStorageClusterVersionModel.java
> the interface '/version/cluster' with header 'Accept: application/json'
> return is not JSON but plain text
> -
>
> Key: HBASE-17713
> URL: https://issues.apache.org/jira/browse/HBASE-17713
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 1.2.2
> Environment: Hhbase 1.2.2
>Reporter: Feng Ce
>Assignee: Guangxu Cheng
>Priority: Minor
> Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-17713.branch-1.patch, HBASE-17713.master.v1.patch,
> HBASE-17713.patch
>
> Original Estimate: 6h
> Remaining Estimate: 6h
>
> Hbase REST API, this interface `get 'version/cluster'`, when I use the header
> `Accept: application/json`, the response is not JSON but plain text.
> {code:none}
> curl -X GET \
> -H "Accept: application/json" \
> "http://localhost:/version/cluster";
>"1.2.2"
> {code}
> But when I use `Accept: text/xml`, the response is correct XML.
> {code:none}
> curl -X GET \
> -H "Accept: text/xml" \
> "http://localhost:/version/cluster";
> standalone="yes"?>1.2.2
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
[ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155953#comment-16155953 ] Hudson commented on HBASE-18375: FAILURE: Integrated in Jenkins build HBase-2.0 #467 (See [https://builds.apache.org/job/HBase-2.0/467/]) HBASE-18375: Fix the bug where the pool chunks from ChunkCreator are (anastas: rev 68ec2a9da022f7824e9a45ef89a0c4b8bcb838f3) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemstoreLABWithoutPool.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreLAB.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionPipeline.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLABImpl.java > The pool chunks from ChunkCreator are deallocated while in pool because there > is no reference to them > - > > Key: HBASE-18375 > URL: https://issues.apache.org/jira/browse/HBASE-18375 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-1 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18375-branch-2.patch, HBASE-18375-V01.patch, > HBASE-18375-V02-branch-2.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch, > HBASE-18375-V04.patch, HBASE-18375-V05.patch, HBASE-18375-V06.patch, > HBASE-18375-V07.patch, HBASE-18375-V08.patch, HBASE-18375-V09.patch, > HBASE-18375-V10.patch, HBASE-18375-V11.patch > > > Because MSLAB list of chunks was changed to list of chunk IDs, the chunks > returned back to pool can be deallocated by JVM because there is no reference > to them. The solution is to protect pool chunks from GC by the strong map of > ChunkCreator introduced by HBASE-18010. Will prepare the patch today. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18737) Display configured max size of memstore and cache on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18737:
---
Resolution: Fixed
Fix Version/s: 1.5.0
1.4.0
Status: Resolved (was: Patch Available)
> Display configured max size of memstore and cache on RS UI
> --
>
> Key: HBASE-18737
> URL: https://issues.apache.org/jira/browse/HBASE-18737
> Project: HBase
> Issue Type: Improvement
>Reporter: Biju Nair
>Assignee: Biju Nair
>Priority: Minor
> Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18737-BRANCH-1.PATCH, HBASE-18737.PATCH,
> HBASE-18737-WIP.PATCH, RS_UI_AFTER-1.PNG, RS_UI_AFTER.PNG, RS_UI_FULL.PNG,
> RS_UI_L1.PNG, RS_UI_L2.PNG, RS_UI.PNG
>
>
> Displaying the configured size of {{memstore}} and {{cache}} will help
> non-admin users understand the cluster capacity. Attached screenshot with
> proposed usability related changes and the current RS UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18751) Revisit the TimeRange and TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155915#comment-16155915 ] stack commented on HBASE-18751: --- I filed HBASE-18767 for alpha-3. > Revisit the TimeRange and TimeRangeTracker > -- > > Key: HBASE-18751 > URL: https://issues.apache.org/jira/browse/HBASE-18751 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18767: -- Fix Version/s: 2.0.0-alpha-3 > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18767: -- Description: From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18751) Revisit the TimeRange and TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155912#comment-16155912 ] Chia-Ping Tsai commented on HBASE-18751: bq. Because only one thread flushing Yup > Revisit the TimeRange and TimeRangeTracker > -- > > Key: HBASE-18751 > URL: https://issues.apache.org/jira/browse/HBASE-18751 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18131) Add an hbase shell command to clear deadserver list in ServerManager
[
https://issues.apache.org/jira/browse/HBASE-18131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155910#comment-16155910
]
Hadoop QA commented on HBASE-18131:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
10s{color} | {color:red} The patch generated 8 new + 347 unchanged - 1 fixed =
355 total (was 348) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m
4s{color} | {color:red} The patch generated 14 new + 726 unchanged - 0 fixed =
740 total (was 726) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
37s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 98m
13s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 18s{color}
| {color:red} hbase-shell in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}173m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestShell |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.
[jira] [Created] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
stack created HBASE-18767: - Summary: Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" Key: HBASE-18767 URL: https://issues.apache.org/jira/browse/HBASE-18767 Project: HBase Issue Type: Task Reporter: stack -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18765) The value of balancerRan is true even though no plans are executed
[
https://issues.apache.org/jira/browse/HBASE-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155909#comment-16155909
]
Hadoop QA commented on HBASE-18765:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
40m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18765 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885645/HBASE-18765.v0.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 05f91eef0d7b 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / fd49a98 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8481/testReport/ |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8481/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> The value of balancerRan is true even though no plans are executed
> --
>
> Key: HBASE-18765
> URL: https://issues.apache.org/jira/browse/HBASE-18765
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>
[jira] [Commented] (HBASE-18751) Revisit the TimeRange and TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155904#comment-16155904 ] Chia-Ping Tsai commented on HBASE-18751: bq. Suggestions for making it more obvious/plain? I had noticed the subject but i get distracted by some other stuff. Can we pin the road map of release on JIRA or somewhere? We don't have a issue like "Release hbase-2.0.0-alpha3" now. Maybe it is time to file the issue. > Revisit the TimeRange and TimeRangeTracker > -- > > Key: HBASE-18751 > URL: https://issues.apache.org/jira/browse/HBASE-18751 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155883#comment-16155883
]
Mikhail Antonov commented on HBASE-16894:
-
conceptually they're solving the same or very similar problem though, aren't
they? seems like one patch here would address the issue?
oh wait, you mean difference is that here it's about live tables, not snapshots?
Regarding my code the problem i faced was (unsure if you or someone else solved
it here or just didn't have it) that when multiple mappers open the same region
they run into concurrency issues as they try to replay WALs and create some
marker files on HDFS. Errors seems harmless but cause mappers to fail.
Was it also problem here?
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Attachments: ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18661) [HLC] Move clocks to a separate o.a.h.h.clocks package
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18661: - Description: --I think we can move all clocks related code out of h-common and h-server into a new module which only needs to depend on h-shaded-protocol. If it's possible, I think it'll be much cleaner code layout and dependency structure.-- Edit: let's just move to a separate package in hbase-common for now. was: --I think we can move all clocks related code out of h-common and h-server into a new package which only needs to depend on h-shaded-protocol. If it's possible, I think it'll be much cleaner code layout and dependency structure.-- Edit: let's just move to a separate package in hbase-common for now. > [HLC] Move clocks to a separate o.a.h.h.clocks package > -- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18661.HBASE-14070.HLC.001.patch > > > --I think we can move all clocks related code out of h-common and h-server > into a new module which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency > structure.-- > Edit: let's just move to a separate package in hbase-common for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155866#comment-16155866
]
Yi Liang commented on HBASE-16894:
--
[~mantonov]
{quote}simple keyspace split as in hbase-18090 by first/end key would be
sufficient for the most needs.{quote}
Agreed.
And I checked your code in HBASE-18090, your changes mainly go to
TableSnapshotInputFormat, and this one is to change TableInputFormatBase. They
are using same same idea, but we still need two jiras to handle them, they are
not a duplicates. And I will first review your code.
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Attachments: ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18751) Revisit the TimeRange and TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155856#comment-16155856 ] stack commented on HBASE-18751: --- On dev list, August 17th, under subject "Moving 2.0 forward", I tried so say what was coming in alpha3. Understood if you missed it. Suggestions for making it more obvious/plain? I'm trying to solidify the API sooner rather than later. Was hoping to nail most of it for alpha3. Its ok if other stuff comes in but API was 'theme'. > Revisit the TimeRange and TimeRangeTracker > -- > > Key: HBASE-18751 > URL: https://issues.apache.org/jira/browse/HBASE-18751 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-10462) Recategorize some of the client facing Public / Private interfaces
[
https://issues.apache.org/jira/browse/HBASE-10462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-10462.
---
Resolution: Fixed
I moved the golden file issue out from under here as a subtask. Resolving as
done.
> Recategorize some of the client facing Public / Private interfaces
> --
>
> Key: HBASE-10462
> URL: https://issues.apache.org/jira/browse/HBASE-10462
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
> Attachments: hbase-10462_wip1.patch
>
>
> We should go over the list of InterfaceAudience.Public interfaces one more to
> remove those that are NOT indeed public interfaces.
> From current trunk, we should change these from public to private:
> {code}
> ReversedScannerCallable
> ReversedClientScanner
> ClientScanner (note that ResultScanner is public interface, while
> ClientScanner should not be)
> ClientSmallScanner
> TableSnapshotScanner -> We need a way of constructing this since it cannot be
> constructed from HConnection / HTable. Maybe a basic factory.
> {code}
> These are not marked:
> {code}
> Registry,
> ZooKeeperRegistry
> RpcRetryingCallerFactory
> ZooKeeperKeepAliveConnection
> AsyncProcess
> DelegatingRetryingCallable
> HConnectionKey
> MasterKeepAliveConnection
> MultiServerCallable
> {code}
> We can think about making these public interface:
> {code}
> ScanMetrics
> {code}
> Add javadoc to:
> {code}
> Query
> {code}
> We can add a test to find out all classes in client package to check for
> interface mark.
> We can extend this to brainstorm on the preferred API options. We probably
> want the clients to use HTableInterface, instead of HTable everywhere.
> HConnectionManager comes with bazillion methods which are not intended for
> public use, etc.
> Raising this as blocker to 1.0
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18765) The value of balancerRan is true even though no plans are executed
[
https://issues.apache.org/jira/browse/HBASE-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18765:
---
Attachment: HBASE-18765.v0.patch
I don't add any tests because the related tests are disabled by HBASE-14614,
and the disabled tests are traced by HBASE-18350.
> The value of balancerRan is true even though no plans are executed
> --
>
> Key: HBASE-18765
> URL: https://issues.apache.org/jira/browse/HBASE-18765
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18765.v0.patch
>
>
> {code}
> //We balance per group instead of per table
> List plans = new ArrayList<>();
> for(Map.Entry>> tableMap:
> getRSGroupAssignmentsByTable(groupName).entrySet()) {
> LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
> + tableMap.getValue());
> List partialPlans =
> balancer.balanceCluster(tableMap.getValue());
> LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
> partialPlans);
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> long startTime = System.currentTimeMillis();
> balancerRan = plans != null;
> {code}
> The *plans* never be null.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18765) The value of balancerRan is true even though no plans are executed
[
https://issues.apache.org/jira/browse/HBASE-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18765:
---
Status: Patch Available (was: Open)
> The value of balancerRan is true even though no plans are executed
> --
>
> Key: HBASE-18765
> URL: https://issues.apache.org/jira/browse/HBASE-18765
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18765.v0.patch
>
>
> {code}
> //We balance per group instead of per table
> List plans = new ArrayList<>();
> for(Map.Entry>> tableMap:
> getRSGroupAssignmentsByTable(groupName).entrySet()) {
> LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
> + tableMap.getValue());
> List partialPlans =
> balancer.balanceCluster(tableMap.getValue());
> LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
> partialPlans);
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> long startTime = System.currentTimeMillis();
> balancerRan = plans != null;
> {code}
> The *plans* never be null.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12556: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-3 > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0-alpha-3 > > Attachments: hbase-12556-wip.patch > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-10462) Recategorize some of the client facing Public / Private interfaces
[
https://issues.apache.org/jira/browse/HBASE-10462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155830#comment-16155830
]
stack commented on HBASE-10462:
---
I'll commit a bit of javadoc for Query. I made HBASE-18766 to do the
TableSnapshotScanner cleanup making it Private.
> Recategorize some of the client facing Public / Private interfaces
> --
>
> Key: HBASE-10462
> URL: https://issues.apache.org/jira/browse/HBASE-10462
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
> Attachments: hbase-10462_wip1.patch
>
>
> We should go over the list of InterfaceAudience.Public interfaces one more to
> remove those that are NOT indeed public interfaces.
> From current trunk, we should change these from public to private:
> {code}
> ReversedScannerCallable
> ReversedClientScanner
> ClientScanner (note that ResultScanner is public interface, while
> ClientScanner should not be)
> ClientSmallScanner
> TableSnapshotScanner -> We need a way of constructing this since it cannot be
> constructed from HConnection / HTable. Maybe a basic factory.
> {code}
> These are not marked:
> {code}
> Registry,
> ZooKeeperRegistry
> RpcRetryingCallerFactory
> ZooKeeperKeepAliveConnection
> AsyncProcess
> DelegatingRetryingCallable
> HConnectionKey
> MasterKeepAliveConnection
> MultiServerCallable
> {code}
> We can think about making these public interface:
> {code}
> ScanMetrics
> {code}
> Add javadoc to:
> {code}
> Query
> {code}
> We can add a test to find out all classes in client package to check for
> interface mark.
> We can extend this to brainstorm on the preferred API options. We probably
> want the clients to use HTableInterface, instead of HTable everywhere.
> HConnectionManager comes with bazillion methods which are not intended for
> public use, etc.
> Raising this as blocker to 1.0
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-12556) Create a golden file for testing client API source/binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12556: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-10462) > Create a golden file for testing client API source/binary compatibility > --- > > Key: HBASE-12556 > URL: https://issues.apache.org/jira/browse/HBASE-12556 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: hbase-12556-wip.patch > > > [~lhofhansl] had a suggestion (in some other jira I forgot) for doing a > golden file for the HBase API so that we can compare between releases to > ensure that we are keeping source and binary compatibility as defined in this > document : > https://docs.google.com/document/d/1p5pP7v2OuzSSaomK2S2v7sfKky1Hex6OqwsJO0sZTUY/edit > > I think we can generate a file, commit it to the repo, and create a unit test > to check whether any API's are broken. Adding new InterfaceAudience.Public > interfaces has to modify this file so that it becomes an explicit decision. > The downside is that we have to modify the file every time we add a new API, > but it should be fine since it will force us to think more before committing > to supporting new interfaces within the same major versions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18444) Add support for specifying custom meta table suffix
[
https://issues.apache.org/jira/browse/HBASE-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155827#comment-16155827
]
Sean Busbey commented on HBASE-18444:
-
FWIW, if they're in a feature fork for the umbrella than I'm fine with things
landing in the branch. Unfortunately not going to have time for another round
of reviewing scope docs, etc for a couple of weeks.
> Add support for specifying custom meta table suffix
> ---
>
> Key: HBASE-18444
> URL: https://issues.apache.org/jira/browse/HBASE-18444
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Labels: patch
> Attachments: HBASE-18444.branch-1.001.patch,
> HBASE-18444.master.001.patch
>
>
> This patch provides user an ability to have a custom meta table suffix.
> In order to share a common rootDir amongst multiple HBase clusters, we need
> separate meta table for each cluster.
> With custom meta table suffix, the meta table is named in the form of
> meta_{suffix} making it specific to a cluster.
> Users can specify the meta table suffix as part of hbase-site.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155826#comment-16155826
]
Mikhail Antonov commented on HBASE-16894:
-
That said, I don't think there's real need to use index info here - simple
keyspace split as in hbase-18090 by first/end key would be sufficient for the
most needs.
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Attachments: ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18766) Make TableSnapshotScanner Audience Private
stack created HBASE-18766: - Summary: Make TableSnapshotScanner Audience Private Key: HBASE-18766 URL: https://issues.apache.org/jira/browse/HBASE-18766 Project: HBase Issue Type: Task Reporter: stack >From HBASE-10462 API cleanup. TableSnapshotScanner should be Audience Private. >It should not be public. See HBASE-10462 for [~enis] rationale. If private, >then there needs to be a way to construct it. It is awkward since we normally >construct 'Table' by getting one from a Connection. Enis suggests: bq. Maybe we can do a SnapshotConnection which returns a ConnectionTable, which returns the scanner. Since connection is the factory to everything, I think this approach is the most clean. Sounds good. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18751) Revisit the TimeRange and TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155823#comment-16155823 ] Chia-Ping Tsai commented on HBASE-18751: bq. Do you think these are alpha3 concerns (theme for alpha3 is API scrubbing) or can they be punted to beta1? Thank you. alpha3 is a good place to go for these issues. BTW, where can i find this announcement about theme? I totally have no idea about that. > Revisit the TimeRange and TimeRangeTracker > -- > > Key: HBASE-18751 > URL: https://issues.apache.org/jira/browse/HBASE-18751 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18697) Need a shaded hbase-mapreduce module
[
https://issues.apache.org/jira/browse/HBASE-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155821#comment-16155821
]
Appy commented on HBASE-18697:
--
Here are the runs
{noformat}
root hbasemr-1 /apache/hadoop (master) → bin/hadoop jar
/apache/hbase/hbase-shaded/hbase-shaded-mapreduce/target/hbase-shaded-mapreduce-3.0.0-SNAPSHOT.jar
org.apache.hadoop.hbase.mapreduce.RowCounter t
Unknown program 'org.apache.hadoop.hbase.mapreduce.RowCounter' chosen.
Valid program names are:
CellCounter: Count cells in HBase table.
WALPlayer: Replay WAL files.
completebulkload: Complete a bulk data load.
copytable: Export a table from local cluster to peer cluster.
export: Write table data to HDFS.
exportsnapshot: Export the specific snapshot to a given FileSystem.
import: Import data written by Export.
importtsv: Import data in TSV format.
rowcounter: Count rows in HBase table.
verifyrep: Compare data from tables in two different clusters. It doesn't
work for incrementColumnValues'd cells since timestamp is changed after
appending to WAL.
{noformat}
{noformat}
root hbasemr-1 /apache/hadoop (master) → bin/hadoop jar
/apache/hbase/hbase-shaded/hbase-shaded-mapreduce/target/hbase-shaded-mapreduce-3.0.0-SNAPSHOT.jar
rowcounter t
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/apache/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/root/.m2/repository/org/slf4j/slf4j-log4j12/1.7.24/slf4j-log4j12-1.7.24.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
17/09/06 11:01:29 INFO Configuration.deprecation: session.id is deprecated.
Instead, use dfs.metrics.session-id
17/09/06 11:01:29 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
17/09/06 11:01:30 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x659eef7 connecting to ZooKeeper
ensemble=hbasemr-1.gce.cloudera.com:2181
17/09/06 11:01:30 INFO impl.MetricsConfig: loaded properties from
hadoop-metrics2-hbase.properties
17/09/06 11:01:30 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10
second(s).
17/09/06 11:01:30 INFO impl.MetricsSystemImpl: HBase metrics system started
17/09/06 11:01:30 INFO metrics.MetricRegistries: Loaded MetricRegistries class
org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
17/09/06 11:01:30 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
17/09/06 11:01:30 INFO zookeeper.ZooKeeper: Client
environment:host.name=hbasemr-1
17/09/06 11:01:30 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.8.0_144
17/09/06 11:01:30 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Oracle Corporation
17/09/06 11:01:30 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/lib/jvm/java-8-oracle/jre
17/09/06 11:01:30 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/apache/hadoop/etc/hadoop:/apache/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/apache/hadoop/share/hadoop/common/lib/slf4j-api-1.7.10.jar:/apache/hadoop/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/apache/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/apache/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/apache/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/apache/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/apache/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/apache/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/apache/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/apache/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/apache/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/apache/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/apache/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/apache/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/apache/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/apache/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/apache/hadoop/share/hadoop/common/lib/junit-4.11.jar:/apache/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/apache/hadoop/share/hadoop/common/lib/curator-client-2.7.1.jar:/apache/hadoop/share/hadoop/common/lib/asm-3.2.jar:/apache/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/apache/hadoop/share/hadoop/common/lib/xz-1.0.jar:/apache/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar:/apache/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/apache/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/apache/hadoop/share/hadoop/common/lib/hadoop-annotations-2.7.1.jar:/apache/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/apache/hadoop/share/hadoop/commo
[jira] [Commented] (HBASE-16894) Create more than 1 split per region, generalize HBASE-12590
[
https://issues.apache.org/jira/browse/HBASE-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155817#comment-16155817
]
Mikhail Antonov commented on HBASE-16894:
-
The reason I wanted to be able to specify numSplits per region is that I wanted
to be able to decouple split policy on one cluster and split policy on another
(with more of analytical workload), where people can export/copy snapshot and
where degree of parallelism may heavily depend on the nature of MR jobs being
run (how CPU / IO intensive are jobs) and size/machine specs on different
clusters.
I think we do want to be able to specify numSplits. No real reason to be super
precise in the split - just enough to provide decent parallelism IMO.
> Create more than 1 split per region, generalize HBASE-12590
> ---
>
> Key: HBASE-16894
> URL: https://issues.apache.org/jira/browse/HBASE-16894
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Attachments: ImplementaionAndSomeQuestion.docx
>
>
> A common request from users is to be able to better control how many map
> tasks are created per region. Right now, it is always 1 region = 1 input
> split = 1 map task. Same goes for Spark since it uses the TIF. With region
> sizes as large as 50 GBs, it is desirable to be able to create more than 1
> split per region.
> HBASE-12590 adds a config property for MR jobs to be able to handle skew in
> region sizes. The algorithm is roughly:
> {code}
> If (region size >= average size*ratio) : cut the region into two MR input
> splits
> If (average size <= region size < average size*ratio) : one region as one MR
> input split
> If (sum of several continuous regions size < average size * ratio): combine
> these regions into one MR input split.
> {code}
> Although we can set data skew ratio to be 0.5 or something to abuse
> HBASE-12590 into creating more than 1 split task per region, it is not ideal.
> But there is no way to create more with the patch as it is. For example we
> cannot create more than 2 tasks per region.
> If we want to fix this properly, we should extend the approach in
> HBASE-12590, and make it so that the client can specify the desired num of
> mappers, or desired split size, and the TIF generates the splits based on the
> current region sizes very similar to the algorithm in HBASE-12590, but a more
> generic way. This also would eliminate the hand tuning of data skew ratio.
> We also can think about the guidepost approach that Phoenix has in the stats
> table which is used for exactly this purpose. Right now, the region can be
> split into powers of two assuming uniform distribution within the region.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18765) The value of balancerRan is true even though no plans are executed
Chia-Ping Tsai created HBASE-18765:
--
Summary: The value of balancerRan is true even though no plans are
executed
Key: HBASE-18765
URL: https://issues.apache.org/jira/browse/HBASE-18765
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Minor
Fix For: 2.0.0-alpha-3
{code}
//We balance per group instead of per table
List plans = new ArrayList<>();
for(Map.Entry>> tableMap:
getRSGroupAssignmentsByTable(groupName).entrySet()) {
LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
+ tableMap.getValue());
List partialPlans =
balancer.balanceCluster(tableMap.getValue());
LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
partialPlans);
if (partialPlans != null) {
plans.addAll(partialPlans);
}
}
long startTime = System.currentTimeMillis();
balancerRan = plans != null;
{code}
The *plans* never be null.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18765) The value of balancerRan is true even though no plans are executed
[
https://issues.apache.org/jira/browse/HBASE-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18765:
---
Component/s: rsgroup
> The value of balancerRan is true even though no plans are executed
> --
>
> Key: HBASE-18765
> URL: https://issues.apache.org/jira/browse/HBASE-18765
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-3
>
>
> {code}
> //We balance per group instead of per table
> List plans = new ArrayList<>();
> for(Map.Entry>> tableMap:
> getRSGroupAssignmentsByTable(groupName).entrySet()) {
> LOG.info("Creating partial plan for table " + tableMap.getKey() + ": "
> + tableMap.getValue());
> List partialPlans =
> balancer.balanceCluster(tableMap.getValue());
> LOG.info("Partial plan for table " + tableMap.getKey() + ": " +
> partialPlans);
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> long startTime = System.currentTimeMillis();
> balancerRan = plans != null;
> {code}
> The *plans* never be null.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18697) Need a shaded hbase-mapreduce module
[ https://issues.apache.org/jira/browse/HBASE-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155801#comment-16155801 ] Appy commented on HBASE-18697: -- You're right. I remember [~stack] added main class to hbase-mapreduce recently. Doing same for hbase-shaded-mapreduce here. See attached 004.patch. Tested on my cluster. Behavior is now similar to hbase-mapreduce jar i.e. job runs when 'rowcounter' is used, but throws help message when class name is specified. > Need a shaded hbase-mapreduce module > > > Key: HBASE-18697 > URL: https://issues.apache.org/jira/browse/HBASE-18697 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Duo Zhang >Assignee: Appy > Fix For: 3.0.0, 2.0.0-alpha-3 > > Attachments: HBASE-18697.master.001.patch, > HBASE-18697.master.002.patch, HBASE-18697.master.003.patch, > HBASE-18697.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18737) Display configured max size of memstore and cache on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155799#comment-16155799
]
Hadoop QA commented on HBASE-18737:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-18737 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18737 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885633/HBASE-18737-BRANCH-1.PATCH
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8480/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Display configured max size of memstore and cache on RS UI
> --
>
> Key: HBASE-18737
> URL: https://issues.apache.org/jira/browse/HBASE-18737
> Project: HBase
> Issue Type: Improvement
>Reporter: Biju Nair
>Assignee: Biju Nair
>Priority: Minor
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18737-BRANCH-1.PATCH, HBASE-18737.PATCH,
> HBASE-18737-WIP.PATCH, RS_UI_AFTER-1.PNG, RS_UI_AFTER.PNG, RS_UI_FULL.PNG,
> RS_UI_L1.PNG, RS_UI_L2.PNG, RS_UI.PNG
>
>
> Displaying the configured size of {{memstore}} and {{cache}} will help
> non-admin users understand the cluster capacity. Attached screenshot with
> proposed usability related changes and the current RS UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18751) Revisit the TimeRange and TimeRangeTracker
[ https://issues.apache.org/jira/browse/HBASE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155798#comment-16155798 ] stack commented on HBASE-18751: --- Sounds good [~chia7712] bq. unsync TimeRangeTracker works for the flush. Because only one thread flushing? Do you think these are alpha3 concerns (theme for alpha3 is API scrubbing) or can they be punted to beta1? Thank you. > Revisit the TimeRange and TimeRangeTracker > -- > > Key: HBASE-18751 > URL: https://issues.apache.org/jira/browse/HBASE-18751 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18697) Need a shaded hbase-mapreduce module
[ https://issues.apache.org/jira/browse/HBASE-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18697: - Attachment: HBASE-18697.master.004.patch > Need a shaded hbase-mapreduce module > > > Key: HBASE-18697 > URL: https://issues.apache.org/jira/browse/HBASE-18697 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Duo Zhang >Assignee: Appy > Fix For: 3.0.0, 2.0.0-alpha-3 > > Attachments: HBASE-18697.master.001.patch, > HBASE-18697.master.002.patch, HBASE-18697.master.003.patch, > HBASE-18697.master.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18737) Display configured max size of memstore and cache on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated HBASE-18737:
--
Attachment: HBASE-18737-BRANCH-1.PATCH
Attaching patch for branch 1. All the changes except display of memstore limit
is included in the patch. RegionServerAccounting in branch-1 doesn't store
memstore limit. [~tedyu], could you please take a look.
> Display configured max size of memstore and cache on RS UI
> --
>
> Key: HBASE-18737
> URL: https://issues.apache.org/jira/browse/HBASE-18737
> Project: HBase
> Issue Type: Improvement
>Reporter: Biju Nair
>Assignee: Biju Nair
>Priority: Minor
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18737-BRANCH-1.PATCH, HBASE-18737.PATCH,
> HBASE-18737-WIP.PATCH, RS_UI_AFTER-1.PNG, RS_UI_AFTER.PNG, RS_UI_FULL.PNG,
> RS_UI_L1.PNG, RS_UI_L2.PNG, RS_UI.PNG
>
>
> Displaying the configured size of {{memstore}} and {{cache}} will help
> non-admin users understand the cluster capacity. Attached screenshot with
> proposed usability related changes and the current RS UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
