[jira] [Updated] (HBASE-21934) RemoteProcedureDispatcher should track the ongoing dispatched calls
[ https://issues.apache.org/jira/browse/HBASE-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21934: - Attachment: HBASE-21934.master.006.patch > RemoteProcedureDispatcher should track the ongoing dispatched calls > --- > > Key: HBASE-21934 > URL: https://issues.apache.org/jira/browse/HBASE-21934 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.x >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Fix For: 3. > > Attachments: HBASE-21934.master.001.patch, > HBASE-21934.master.002.patch, HBASE-21934.master.003.patch, > HBASE-21934.master.004.patch, HBASE-21934.master.005.patch, > HBASE-21934.master.006.patch > > > I encounter the problem that when master assign a splitWALRemoteProcedure to > a region server. The log of this region server says it failed to recover the > lease of this file. Then this region server is killed by chaosMonkey. As the > result, this procedure is not timeout and hang there forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1610#comment-1610 ] Yi Mei commented on HBASE-21820: The failed UT are not related, attach the patch and try again > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1616#comment-1616
]
Hadoop QA commented on HBASE-21481:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
18s{color} | {color:blue} hbase-server in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 3s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
10s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}261m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}309m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFastFail |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.client.TestFromClientSide3 |
| | hadoop.hbase.master.procedure.TestServerCrashProcedureWithReplicas |
| | hadoop.hbase.master.TestAssignmentManagerMetrics |
| | hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21481 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960117/HBASE-21481.master.011.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck
[jira] [Updated] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21820: --- Fix Version/s: 2.3.0 2.2.0 3.0.0 > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[
https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1640#comment-1640
]
Guanghao Zhang commented on HBASE-21487:
{code:java}
if (shouldCheckDescriptor) {
2653submitProcedure(new
ModifyTableProcedure(procedureExecutor.getEnvironment(),
2654newDescriptor, latch, oldDescriptor, shouldCheckDescriptor));
2655} else {
2656submitProcedure(
2657new ModifyTableProcedure(procedureExecutor.getEnvironment(),
newDescriptor, latch));
2658}
{code}
submitProcedure( new ModifyTableProcedure(procedureExecutor.getEnvironment(),
newDescriptor, latch, shouldCheckDescriptor)); directly? No need to check
shouldCheckDescriptor.
> Concurrent modify table ops can lead to unexpected results
> --
>
> Key: HBASE-21487
> URL: https://issues.apache.org/jira/browse/HBASE-21487
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Syeda Arshiya Tabreen
>Assignee: Syeda Arshiya Tabreen
>Priority: Major
> Fix For: 2.2.0
>
> Attachments: HBASE-21487.branch-2.02.patch,
> HBASE-21487.branch-2.03.patch, HBASE-21487.branch-2.04.patch,
> HBASE-21487.branch-2.05.patch, HBASE-21487.branch-2.patch
>
>
> Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect
> result. After HBASE-18893, The behavior of add/delete/modify column family
> during concurrent operation is changed compare to branch-1.When one client
> is adding cf2 and another one cf3 .. In branch-1 final result will be
> cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome depending
> on which ModifyTableProcedure executed finally.Its because new table
> descriptor is constructed before submitting the ModifyTableProcedure in
> HMaster class and its not guarded by any lock.
> *Steps to reproduce*
> 1.Create table 't' with column family 'f1'
> 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table
> 't' concurrently.
> *Expected Result*
> Table should have three column families(f1,f2,f3)
> *Actual Result*
> Table 't' will have column family either (f1,f2) or (f1,f3)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21958) Make CLUSTER scope quota work right when use rs group
Yi Mei created HBASE-21958: -- Summary: Make CLUSTER scope quota work right when use rs group Key: HBASE-21958 URL: https://issues.apache.org/jira/browse/HBASE-21958 Project: HBase Issue Type: Sub-task Reporter: Yi Mei HBASE-21820 implement CLUSTER scope quota in a simple way, use [ClusterLimit / RSNum] to divide cluster limit into machine limit. But when use rs group feature, namespace tables are on a subset of region servers, the cluster limit should be shared by the rs group, not all region servers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21487) Concurrent modify table ops can lead to unexpected results
[
https://issues.apache.org/jira/browse/HBASE-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1647#comment-1647
]
Guanghao Zhang commented on HBASE-21487:
{code:java}
preflightChecks(env, null/* No table checks; if changing peers, table can be
online */);
this.modifiedTableDescriptor = newTableDescriptor;
{code}
The preflightChecks should after this.modifiedTableDescriptor =
newTableDescriptor; As the check will call getTableName and it need
modifiedTableDescriptor.
{code:java}
private void initilize() {code}
Add two parameter unmodifiedTableDescriptor and shouldCheckDescriptor for it?
And no need to assign them again.
> Concurrent modify table ops can lead to unexpected results
> --
>
> Key: HBASE-21487
> URL: https://issues.apache.org/jira/browse/HBASE-21487
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Syeda Arshiya Tabreen
>Assignee: Syeda Arshiya Tabreen
>Priority: Major
> Fix For: 2.2.0
>
> Attachments: HBASE-21487.branch-2.02.patch,
> HBASE-21487.branch-2.03.patch, HBASE-21487.branch-2.04.patch,
> HBASE-21487.branch-2.05.patch, HBASE-21487.branch-2.patch
>
>
> Concurrent modifyTable or add/delete/modify columnFamily leads to incorrect
> result. After HBASE-18893, The behavior of add/delete/modify column family
> during concurrent operation is changed compare to branch-1.When one client
> is adding cf2 and another one cf3 .. In branch-1 final result will be
> cf1,cf2,cf3 but now either cf1,cf2 OR cf1,cf3 will be the outcome depending
> on which ModifyTableProcedure executed finally.Its because new table
> descriptor is constructed before submitting the ModifyTableProcedure in
> HMaster class and its not guarded by any lock.
> *Steps to reproduce*
> 1.Create table 't' with column family 'f1'
> 2.Client-1 and Client-2 requests to add column family 'f2' and 'f3' on table
> 't' concurrently.
> *Expected Result*
> Table should have three column families(f1,f2,f3)
> *Actual Result*
> Table 't' will have column family either (f1,f2) or (f1,f3)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21916) Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool
[
https://issues.apache.org/jira/browse/HBASE-21916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1676#comment-1676
]
Hadoop QA commented on HBASE-21916:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-21879 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
6s{color} | {color:green} HBASE-21879 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
12s{color} | {color:green} HBASE-21879 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
13s{color} | {color:green} HBASE-21879 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} HBASE-21879 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} HBASE-21879 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} hbase-common: The patch generated 0 new + 104
unchanged - 2 fixed = 104 total (was 106) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} hbase-server: The patch generated 0 new + 46
unchanged - 5 fixed = 46 total (was 51) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 21s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}259m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}319m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ha
[jira] [Commented] (HBASE-21947) TestShell is broken after we remove the jackson dependencies
[
https://issues.apache.org/jira/browse/HBASE-21947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777821#comment-16777821
]
Peter Somogyi commented on HBASE-21947:
---
The last 5 runs passed on the flaky dashboard.
> TestShell is broken after we remove the jackson dependencies
>
>
> Key: HBASE-21947
> URL: https://issues.apache.org/jira/browse/HBASE-21947
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21947-v1.patch, HBASE-21947.patch
>
>
> {noformat}
> Error: test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
> NameError: cannot load Java class com.fasterxml.jackson.databind.ObjectMapper
> org/jruby/javasupport/JavaClass.java:286:in `for_name'
> org/jruby/javasupport/JavaUtilities.java:34:in `get_proxy_class'
> uri:classloader:/jruby/java/core_ext/object.rb:49:in `block in java_import'
> org/jruby/RubyArray.java:2486:in `map'
> uri:classloader:/jruby/java/core_ext/object.rb:36:in `java_import'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:77:in
> `tasksOnHost'
> src/test/ruby/hbase/taskmonitor_test.rb:33:in `block in
> test_tasksOnHost_should_return_tasks_list'
> 30: filter = 'all'
> 31: hosts = admin.getRegionServers()
> 32: hosts.each do |host|
> => 33: tasks = taskmonitor.tasksOnHost(filter,host)
> 34: assert(tasks.length > 0)
> 35: end
> 36: end
> org/jruby/RubyArray.java:1734:in `each'
> src/test/ruby/hbase/taskmonitor_test.rb:32:in `block in
> test_tasksOnHost_should_return_tasks_list'
> {noformat}
> Seems we are using jackson in hbase-shell.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777838#comment-16777838
]
Reid Chan commented on HBASE-21481:
---
Failed tests passed on local with patch applied.
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch,
> HBASE-21481.master.006.patch, HBASE-21481.master.007.patch,
> HBASE-21481.master.008.patch, HBASE-21481.master.009.patch,
> HBASE-21481.master.010.patch, HBASE-21481.master.011.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21934) RemoteProcedureDispatcher should track the ongoing dispatched calls
[
https://issues.apache.org/jira/browse/HBASE-21934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777839#comment-16777839
]
Hadoop QA commented on HBASE-21934:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
28s{color} | {color:blue} hbase-server in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
20s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}160m
18s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}219m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21934 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960151/HBASE-21934.master.006.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1fc2414a72b9 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon Sep
24 17:14:57 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c33ceb23d3 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf4
[jira] [Commented] (HBASE-21947) TestShell is broken after we remove the jackson dependencies
[
https://issues.apache.org/jira/browse/HBASE-21947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777856#comment-16777856
]
Duo Zhang commented on HBASE-21947:
---
Good. Then let me commit to branch-2 and branch-2.2.
Thanks [~psomogyi].
> TestShell is broken after we remove the jackson dependencies
>
>
> Key: HBASE-21947
> URL: https://issues.apache.org/jira/browse/HBASE-21947
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21947-v1.patch, HBASE-21947.patch
>
>
> {noformat}
> Error: test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
> NameError: cannot load Java class com.fasterxml.jackson.databind.ObjectMapper
> org/jruby/javasupport/JavaClass.java:286:in `for_name'
> org/jruby/javasupport/JavaUtilities.java:34:in `get_proxy_class'
> uri:classloader:/jruby/java/core_ext/object.rb:49:in `block in java_import'
> org/jruby/RubyArray.java:2486:in `map'
> uri:classloader:/jruby/java/core_ext/object.rb:36:in `java_import'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:77:in
> `tasksOnHost'
> src/test/ruby/hbase/taskmonitor_test.rb:33:in `block in
> test_tasksOnHost_should_return_tasks_list'
> 30: filter = 'all'
> 31: hosts = admin.getRegionServers()
> 32: hosts.each do |host|
> => 33: tasks = taskmonitor.tasksOnHost(filter,host)
> 34: assert(tasks.length > 0)
> 35: end
> 36: end
> org/jruby/RubyArray.java:1734:in `each'
> src/test/ruby/hbase/taskmonitor_test.rb:32:in `block in
> test_tasksOnHost_should_return_tasks_list'
> {noformat}
> Seems we are using jackson in hbase-shell.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21947) TestShell is broken after we remove the jackson dependencies
[
https://issues.apache.org/jira/browse/HBASE-21947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21947:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch-2.2+.
Thanks [~psomogyi].
> TestShell is broken after we remove the jackson dependencies
>
>
> Key: HBASE-21947
> URL: https://issues.apache.org/jira/browse/HBASE-21947
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21947-v1.patch, HBASE-21947.patch
>
>
> {noformat}
> Error: test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
> NameError: cannot load Java class com.fasterxml.jackson.databind.ObjectMapper
> org/jruby/javasupport/JavaClass.java:286:in `for_name'
> org/jruby/javasupport/JavaUtilities.java:34:in `get_proxy_class'
> uri:classloader:/jruby/java/core_ext/object.rb:49:in `block in java_import'
> org/jruby/RubyArray.java:2486:in `map'
> uri:classloader:/jruby/java/core_ext/object.rb:36:in `java_import'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:77:in
> `tasksOnHost'
> src/test/ruby/hbase/taskmonitor_test.rb:33:in `block in
> test_tasksOnHost_should_return_tasks_list'
> 30: filter = 'all'
> 31: hosts = admin.getRegionServers()
> 32: hosts.each do |host|
> => 33: tasks = taskmonitor.tasksOnHost(filter,host)
> 34: assert(tasks.length > 0)
> 35: end
> 36: end
> org/jruby/RubyArray.java:1734:in `each'
> src/test/ruby/hbase/taskmonitor_test.rb:32:in `block in
> test_tasksOnHost_should_return_tasks_list'
> {noformat}
> Seems we are using jackson in hbase-shell.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21820) Implement CLUSTER quota scope
[
https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777887#comment-16777887
]
Hadoop QA commented on HBASE-21820:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
59s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
22s{color} | {color:blue} hbase-server in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 55s{color}
| {color:red} hbase-server generated 2 new + 186 unchanged - 2 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
5s{color} | {color:red} hbase-server: The patch generated 3 new + 11 unchanged
- 1 fixed = 14 total (was 12) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
9s{color} | {color:red} The patch generated 16 new + 133 unchanged - 9 fixed =
149 total (was 142) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 10s{color} | {color:orange} The patch generated 46 new + 433 unchanged - 0
fixed = 479 total (was 433) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 58s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
18s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}249m 24s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
48s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}312m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | hadoop.hbase.master.proced
[jira] [Commented] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777890#comment-16777890 ] Guanghao Zhang commented on HBASE-21820: Seems the same ut failed again?[~Yi Mei] > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20724) Sometimes some compacted storefiles are still opened after region failover
[ https://issues.apache.org/jira/browse/HBASE-20724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-20724: --- Attachment: HBASE-20724.master.013.patch Reattach for HADOOP QA. > Sometimes some compacted storefiles are still opened after region failover > -- > > Key: HBASE-20724 > URL: https://issues.apache.org/jira/browse/HBASE-20724 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Guanghao Zhang >Priority: Critical > Attachments: HBASE-20724.master.001.patch, > HBASE-20724.master.002.patch, HBASE-20724.master.003.patch, > HBASE-20724.master.004.patch, HBASE-20724.master.005.patch, > HBASE-20724.master.006.patch, HBASE-20724.master.007.patch, > HBASE-20724.master.008.patch, HBASE-20724.master.009.patch, > HBASE-20724.master.010.patch, HBASE-20724.master.011.patch, > HBASE-20724.master.012.patch, HBASE-20724.master.013.patch, > HBASE-20724.master.013.patch > > > It is important that compacted storefiles of a given compaction execution are > wholly opened or archived to insure data consistency. ie a storefile > containing delete tombstones can be archived while older storefiles > containing cells that were supposed to be deleted are left unarchived thereby > undeleting those cells. > When a server fails compaction markers (in the wal edit) are used to > determine which storefiles are compacted and should be excluded during region > open (during failover). But the WALs containing compaction markers can be > prematurely archived even though there are still compacted storefiles for > that particular compaction event that hasn't been archived yet. Thus losing > compaction information that needs to be replayed in the event of an RS crash. > This is because hlog archiving logic only keeps track of flushed storefiles > and not compacted ones. > https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21717) Implement Connection based on AsyncConnection
[
https://issues.apache.org/jira/browse/HBASE-21717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777897#comment-16777897
]
Hadoop QA commented on HBASE-21717:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 69 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-21512 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
6s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
4s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
20s{color} | {color:blue} hbase-server in HBASE-21512 has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
56s{color} | {color:green} HBASE-21512 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 44s{color}
| {color:red} hbase-client generated 6 new + 85 unchanged - 6 fixed = 91 total
(was 91) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 5s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} hbase-client: The patch generated 1 new + 184
unchanged - 32 fixed = 185 total (was 216) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
15s{color} | {color:red} hbase-server: The patch generated 7 new + 801
unchanged - 53 fixed = 808 total (was 854) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch passed checkstyle in hbase-mapreduce
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} The patch passed checkstyle in hbase-thrift {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-backup: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-rest: The patch generated 1 new + 77 unchanged -
59 fixed = 78 total (was 136) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 43s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
13s{colo
[jira] [Created] (HBASE-21959) CompactionTool should close the store it uses for compacting files, in order to properly archive compacted files.
Wellington Chevreuil created HBASE-21959: Summary: CompactionTool should close the store it uses for compacting files, in order to properly archive compacted files. Key: HBASE-21959 URL: https://issues.apache.org/jira/browse/HBASE-21959 Project: HBase Issue Type: Bug Components: tooling Reporter: Wellington Chevreuil While using CompactionTool to offload RSes, noticed compacted files were never archived from original region dir, causing the space used by the region to actually double. Going through its compaction related code on HStore, which is used by CompactionTool for performing compactions, found out what that compacted files archiving happens mainly while closing the HStore instance. CompactionTool is never explicitly closing its HStore instance, so adding a simple patch that properly close the store. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21959) CompactionTool should close the store it uses for compacting files, in order to properly archive compacted files.
[ https://issues.apache.org/jira/browse/HBASE-21959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil reassigned HBASE-21959: Assignee: Wellington Chevreuil > CompactionTool should close the store it uses for compacting files, in order > to properly archive compacted files. > - > > Key: HBASE-21959 > URL: https://issues.apache.org/jira/browse/HBASE-21959 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > > While using CompactionTool to offload RSes, noticed compacted files were > never archived from original region dir, causing the space used by the region > to actually double. Going through its compaction related code on HStore, > which is used by CompactionTool for performing compactions, found out what > that compacted files archiving happens mainly while closing the HStore > instance. CompactionTool is never explicitly closing its HStore instance, so > adding a simple patch that properly close the store. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21959) CompactionTool should close the store it uses for compacting files, in order to properly archive compacted files.
[ https://issues.apache.org/jira/browse/HBASE-21959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-21959: - Attachment: HBASE-21959-master-001.patch > CompactionTool should close the store it uses for compacting files, in order > to properly archive compacted files. > - > > Key: HBASE-21959 > URL: https://issues.apache.org/jira/browse/HBASE-21959 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Attachments: HBASE-21959-master-001.patch > > > While using CompactionTool to offload RSes, noticed compacted files were > never archived from original region dir, causing the space used by the region > to actually double. Going through its compaction related code on HStore, > which is used by CompactionTool for performing compactions, found out what > that compacted files archiving happens mainly while closing the HStore > instance. CompactionTool is never explicitly closing its HStore instance, so > adding a simple patch that properly close the store. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21804) Remove 0.94 check from the Linkchecker job
[ https://issues.apache.org/jira/browse/HBASE-21804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-21804: -- Resolution: Not A Problem Status: Resolved (was: Patch Available) 0.94 documentation was removed from the website. > Remove 0.94 check from the Linkchecker job > -- > > Key: HBASE-21804 > URL: https://issues.apache.org/jira/browse/HBASE-21804 > Project: HBase > Issue Type: Sub-task >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Attachments: hbase-21804.master.001.patch > > > This is a pretty old release. Even though we don't have the link to the doc > from our main page, the linkchecker job lands directly at > [https://hbase.apache.org/0.94/] which has around 90 odd missing file issues. > I haven't yet looked at the missing anchors stuff yet. > We can set linkchecker to not check 0.94. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-10943) Backport HBASE-7329 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-10943: -- Resolution: Won't Fix Status: Resolved (was: Patch Available) 0.94 release line is retired. > Backport HBASE-7329 to 0.94 > --- > > Key: HBASE-10943 > URL: https://issues.apache.org/jira/browse/HBASE-10943 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.18 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: HBASE-10943-0.94-v1.diff > > > See HBASE-7329 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-8526) Backport HBASE-5667 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-8526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-8526. -- Resolution: Won't Fix 0.94 release line is retired. > Backport HBASE-5667 into 0.94 > - > > Key: HBASE-8526 > URL: https://issues.apache.org/jira/browse/HBASE-8526 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 0.94.7 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Attachments: HBASE-8526-v1-0.94.patch, HBASE-8526-v2-0.94.patch, > HBASE-8526-v3-0.94.patch > > > RegexStringComparator supports java.util.regex.Pattern flags > https://issues.apache.org/jira/browse/HBASE-5667 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-9360) Enable 0.94 -> 0.96 replication to minimize upgrade down time
[ https://issues.apache.org/jira/browse/HBASE-9360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-9360. -- Resolution: Won't Fix 0.94 release line is retired. > Enable 0.94 -> 0.96 replication to minimize upgrade down time > - > > Key: HBASE-9360 > URL: https://issues.apache.org/jira/browse/HBASE-9360 > Project: HBase > Issue Type: Brainstorming > Components: migration >Affects Versions: 0.98.0, 0.96.0 >Reporter: Jeffrey Zhong >Priority: Major > > As we know 0.96 is a singularity release, as of today a 0.94 hbase user has > to do in-place upgrade: make corresponding client changes, recompile client > application code, fully shut down existing 0.94 hbase cluster, deploy 0.96 > binary, run upgrade script and then start the upgraded cluster. You can image > the down time will be extended if something is wrong in between. > To minimize the down time, another possible way is to setup a secondary 0.96 > cluster and then setup replication between the existing 0.94 cluster and the > new 0.96 slave cluster. Once the 0.96 cluster is synced, a user can switch > the traffic to the 0.96 cluster and decommission the old one. > The ideal steps will be: > 1) Setup a 0.96 cluster > 2) Setup replication between a running 0.94 cluster to the newly created 0.96 > cluster > 3) Wait till they're in sync in replication > 4) Starts duplicated writes to both 0.94 and 0.96 clusters(could stop > relocation now) > 5) Forward read traffic to the slave 0.96 cluster > 6) After a certain period, stop writes to the original 0.94 cluster if > everything is good and completes upgrade > To get us there, there are two tasks: > 1) Enable replication from 0.94 -> 0.96 > I've run the idea with [~jdcryans], [~devaraj] and [~ndimiduk]. Currently it > seems the best approach is to build a very similar service or on top of > https://github.com/NGDATA/hbase-indexer/tree/master/hbase-sep with support > three commands replicateLogEntries, multi and delete. Inside the three > commands, we just pass down the corresponding requests to the destination > 0.96 cluster as a bridge. The reason to support the multi and delete is for > CopyTable to copy data from a 0.94 cluster to a 0.96 one. > The other approach is to provide limited support of 0.94 RPC protocol in > 0.96. While an issue on this is that a 0.94 client needs to talk to zookeeper > firstly before it can connect to a 0.96 region server. Therefore, we need a > faked Zookeeper setup in front of a 0.96 cluster for a 0.94 client to > connect. It may also pollute 0.96 code base with 0.94 RPC code. > 2) To support writes to a 0.96 cluster and a 0.94 at the same time, we need > to load both hbase clients into one single JVM using different class loader. > Let me know if you think this is worth to do and any better approach we could > take. > Thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-8079) Mismatch of mutateRow() parameters between DemoClient.php and gen-php/Hbase.php in 0.94
[
https://issues.apache.org/jira/browse/HBASE-8079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-8079.
--
Resolution: Won't Fix
0.94 release line is retired.
> Mismatch of mutateRow() parameters between DemoClient.php and
> gen-php/Hbase.php in 0.94
> ---
>
> Key: HBASE-8079
> URL: https://issues.apache.org/jira/browse/HBASE-8079
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.2
>Reporter: Ted Yu
>Priority: Major
>
> From user Jung under email thread 'Hbase Thrift DemoClient.php bug'
> This appears to be 0.94 specific.
> --
> installed thrift and hbase. and testing php with hbase.
> i tested apache thrift 0.90 and 1.0-dev with hbase 0.94.2 and hbase
> 0.94.5 's Hbase.thrift
> $ php DemoClient.php failed like below
> {code}
>
>
> DemoClient
>
>
>
> scanning tables...
> found: cars
> found: demo_table
> disabling table: demo_table
> deleting table: demo_table
> found: dnstest
> found: hello_world
> creating table: demo_table
> column families in demo_table:
> column: entry:, maxVer: 10
> column: unused:, maxVer: 3
> PHP Warning: Missing argument 4 for Hbase\HbaseClient::mutateRow(),
> called in /home/hbase/test/www-current/DemoClient.php on line 138 and
> defined in /home/hbase/test/www-current/thrift/packages/Hbase/Hbase.php
> on line 1233
> PHP Notice: Undefined variable: attributes in
> /home/hbase/test/www-current/thrift/packages/Hbase/Hbase.php on line
> 1235
> PHP Warning: Missing argument 4 for Hbase\HbaseClient::mutateRow(),
> called in /home/hbase/test/www-current/DemoClient.php on line 147 and
> defined in /home/hbase/test/www-current/thrift/packages/Hbase/Hbase.php
> on line 1233
> {code}
> ---
> DemoClient.php using 3 args
> $client->mutateRow( $t, "foo", $mutations )
> but mutateRow in gen-php/Hbase.php have a 4 args like below
> {code}
> 40: public function mutateRow($tableName, $row, $mutations, $attributes);
> 41: public function mutateRowTs($tableName, $row, $mutations,
> $timestamp, $attributes);
> 42: public function mutateRows($tableName, $rowBatches, $attributes);
> 43: public function mutateRowsTs($tableName, $rowBatches, $timestamp,
> $attributes);
> 1233: public function mutateRow($tableName, $row, $mutations, $attributes)
> 1290: public function mutateRowTs($tableName, $row, $mutations,
> $timestamp, $attributes)
> 1348: public function mutateRows($tableName, $rowBatches, $attributes)
> 1404: public function mutateRowsTs($tableName, $rowBatches,
> $timestamp, $attributes)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-10980) Backport to 0.94 "HBASE-10543 Two rare test failures with TestLogsCleaner and TestSplitLogWorker"
[ https://issues.apache.org/jira/browse/HBASE-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-10980. --- Resolution: Won't Fix 0.94 release line is retired. > Backport to 0.94 "HBASE-10543 Two rare test failures with TestLogsCleaner and > TestSplitLogWorker" > - > > Key: HBASE-10980 > URL: https://issues.apache.org/jira/browse/HBASE-10980 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Attachments: 10980.txt > > > I saw this test fail in an internal test rig against 0.94 so backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-5711) Tests are failing with incorrect data directory permissions.
[
https://issues.apache.org/jira/browse/HBASE-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-5711.
--
Resolution: Won't Fix
0.94 release line is retired.
> Tests are failing with incorrect data directory permissions.
>
>
> Key: HBASE-5711
> URL: https://issues.apache.org/jira/browse/HBASE-5711
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
>Priority: Major
> Attachments: HBASE-5711.patch
>
>
> When we run some tests in Hbase (TestAdmin), it is failing with following
> error.
> {quote}
> Starting DataNode 0 with dfs.data.dir:
> E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb\dfs\data\data1,E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb\dfs\data\data2
> 2012-04-04 18:04:51,036 WARN [main] impl.MetricsSystemImpl(137): Metrics
> system not started: Cannot locate configuration: tried
> hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
> 2012-04-04 18:04:51,255 WARN [main] datanode.DataNode(1548): Invalid
> directory in dfs.data.dir: Incorrect permission for
> E:/Repositories/Hbase/target/test-data/5ff23198-892e-4f1c-8022-b3d9969fcf0b/dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb/dfs/data/data1,
> expected: rwxr-xr-x, while actual: rwx--
> 2012-04-04 18:04:51,411 WARN [main] datanode.DataNode(1548): Invalid
> directory in dfs.data.dir: Incorrect permission for
> E:/Repositories/Hbase/target/test-data/5ff23198-892e-4f1c-8022-b3d9969fcf0b/dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb/dfs/data/data2,
> expected: rwxr-xr-x, while actual: rwx--
> 2012-04-04 18:04:51,411 ERROR [main] datanode.DataNode(1554): All directories
> in dfs.data.dir are invalid.
> 2012-04-04 18:04:51,411 INFO [main] hbase.HBaseTestingUtility(684): Shutting
> down minicluster
> 2012-04-04 18:04:51,646 WARN [main] hbase.HBaseTestingUtility(696): Failed
> delete of
> E:\Repositories\Hbase\target\test-data\5ff23198-892e-4f1c-8022-b3d9969fcf0b\dfscluster_0ecc6984-1925-4870-ac7c-439fceede4cb
> 2012-04-04 18:04:51,646 INFO [main] hbase.HBaseTestingUtility(700):
> Minicluster is down
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-7150) Utility class to determine File Descriptor counts depending on the JVM Vendor
[ https://issues.apache.org/jira/browse/HBASE-7150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-7150. -- Resolution: Won't Fix > Utility class to determine File Descriptor counts depending on the JVM Vendor > - > > Key: HBASE-7150 > URL: https://issues.apache.org/jira/browse/HBASE-7150 > Project: HBase > Issue Type: Sub-task > Components: build, util >Affects Versions: 0.94.6, 0.95.0 > Environment: Non Sun JDK environments >Reporter: Kumar Ravi >Assignee: Kumar Ravi >Priority: Major > Attachments: HBASE-7150.patch > > > This issue is being opened to replace the OSMXBean class that was submitted > in HBASE-6965. A new utility class called JVM is being opened to be used by > ResourceChecker (0.94 branch) and ResourceCheckerJUnitListener test classes. > The patch for the ResourceChecker classes is being addressed by HBASE-6945. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-12649) [0.94] port HBASE-11705 callQueueSize should be decremented in a fail-fast scenario
[ https://issues.apache.org/jira/browse/HBASE-12649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-12649. --- Resolution: Won't Fix 0.94 release line is retired. > [0.94] port HBASE-11705 callQueueSize should be decremented in a fail-fast > scenario > --- > > Key: HBASE-12649 > URL: https://issues.apache.org/jira/browse/HBASE-12649 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: hongyu bi >Priority: Major > > Refer to HBASE-11705 which may lead to "callQueueSize leak" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-9568) backport HBASE-6508 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-9568. -- Resolution: Won't Fix 0.94 release line is retired. > backport HBASE-6508 to 0.94 > --- > > Key: HBASE-9568 > URL: https://issues.apache.org/jira/browse/HBASE-9568 > Project: HBase > Issue Type: Improvement > Components: MTTR >Reporter: Liu Shaohui >Priority: Minor > Attachments: HBASE-9568-0.94-v1.diff > > > Backport HBASE-6508: Filter out edits at log split time to hbase 0.94 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21416) Intermittent TestRegionInfoDisplay failure due to shift in relTime of RegionState#toDescriptiveString
[
https://issues.apache.org/jira/browse/HBASE-21416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777965#comment-16777965
]
Norbert Kalmar commented on HBASE-21416:
I took a look on this ticket.
The test has a private method to check discriptiveName with a function that
keeps in mind millisecond can differ:
checkDescriptiveNameEquality()
It is used for the first comparison, but not for the second.
I think this should solve the fkaliness (I couldn't reproduce the flaky test
nor does Jenkins flaky runs show any failure lately, but looking at the jiras
there were some instances when this failed. Rarely though.)
I will create a patch that can be reviewed.
> Intermittent TestRegionInfoDisplay failure due to shift in relTime of
> RegionState#toDescriptiveString
> -
>
> Key: HBASE-21416
> URL: https://issues.apache.org/jira/browse/HBASE-21416
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>
> Over
> https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2.1/1799/testReport/junit/org.apache.hadoop.hbase.client/TestRegionInfoDisplay/testRegionDetailsForDisplay/
> :
> {code}
> org.junit.ComparisonFailure: expected:<...:30 UTC 2018 (PT0.00[6]S ago),
> server=null> but was:<...:30 UTC 2018 (PT0.00[7]S ago), server=null>
> at
> org.apache.hadoop.hbase.client.TestRegionInfoDisplay.testRegionDetailsForDisplay(TestRegionInfoDisplay.java:78)
> {code}
> Here is how toDescriptiveString composes relTime:
> {code}
> long relTime = System.currentTimeMillis() - stamp;
> {code}
> In the test, state.toDescriptiveString() is called twice for the assertion
> where different return values from System.currentTimeMillis() caused the
> assertion to fail in the above occasion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-13019) Fork 0.98 docs after HBASE-12585 ?
[ https://issues.apache.org/jira/browse/HBASE-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-13019. --- Resolution: Incomplete > Fork 0.98 docs after HBASE-12585 ? > --- > > Key: HBASE-13019 > URL: https://issues.apache.org/jira/browse/HBASE-13019 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Priority: Major > > Should we fork the 0.98 docs like we have with 0.94 after HBASE-12585 ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21416) Intermittent TestRegionInfoDisplay failure due to shift in relTime of RegionState#toDescriptiveString
[
https://issues.apache.org/jira/browse/HBASE-21416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Norbert Kalmar reassigned HBASE-21416:
--
Assignee: Norbert Kalmar
> Intermittent TestRegionInfoDisplay failure due to shift in relTime of
> RegionState#toDescriptiveString
> -
>
> Key: HBASE-21416
> URL: https://issues.apache.org/jira/browse/HBASE-21416
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Norbert Kalmar
>Priority: Minor
>
> Over
> https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2.1/1799/testReport/junit/org.apache.hadoop.hbase.client/TestRegionInfoDisplay/testRegionDetailsForDisplay/
> :
> {code}
> org.junit.ComparisonFailure: expected:<...:30 UTC 2018 (PT0.00[6]S ago),
> server=null> but was:<...:30 UTC 2018 (PT0.00[7]S ago), server=null>
> at
> org.apache.hadoop.hbase.client.TestRegionInfoDisplay.testRegionDetailsForDisplay(TestRegionInfoDisplay.java:78)
> {code}
> Here is how toDescriptiveString composes relTime:
> {code}
> long relTime = System.currentTimeMillis() - stamp;
> {code}
> In the test, state.toDescriptiveString() is called twice for the assertion
> where different return values from System.currentTimeMillis() caused the
> assertion to fail in the above occasion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-3680) Publish more metrics about mslab
[ https://issues.apache.org/jira/browse/HBASE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-3680. -- Resolution: Won't Fix > Publish more metrics about mslab > > > Key: HBASE-3680 > URL: https://issues.apache.org/jira/browse/HBASE-3680 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.1 >Reporter: Jean-Daniel Cryans >Priority: Major > Attachments: hbase-3680.txt, hbase-3680.txt > > > We have been using mslab on all our clusters for a while now and it seems it > tends to OOME or send us into GC loops of death a lot more than it used to. > For example, one RS with mslab enabled and 7GB of heap died out of OOME this > afternoon; it had .55GB in the block cache and 2.03GB in the memstores which > doesn't account for much... but it could be that because of mslab a lot of > space was lost in those incomplete 2MB blocks and without metrics we can't > really tell. Compactions were running at the time of the OOME and I see block > cache activity. The average load on that cluster is 531. > We should at least publish the total size of all those blocks and maybe even > take actions based on that (like force flushing). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777974#comment-16777974
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #5 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/5/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/5//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/5//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/5//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777981#comment-16777981
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #116 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/116/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/116//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/116//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/116//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/116//console].
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-18299) TestCompactionInDeadRegionServer is flaky
[ https://issues.apache.org/jira/browse/HBASE-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-18299. --- Resolution: Cannot Reproduce Not on any of the flaky lists. > TestCompactionInDeadRegionServer is flaky > - > > Key: HBASE-18299 > URL: https://issues.apache.org/jira/browse/HBASE-18299 > Project: HBase > Issue Type: Test >Affects Versions: 1.4.0, 2.0.0 >Reporter: Phil Yang >Priority: Major > > In this test, when master marks a RS dead and rename its WAL directory, we > expect the compaction throwing an exception. However, if RS finds the WAL is > not writable it will close itself and mark HRegion#areWritesEnabled false and > in compaction it will just skip and doesn't throw any exception. > See > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L1644 > and > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L1964 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19715) Fix timing out test TestMultiRespectsLimits
[ https://issues.apache.org/jira/browse/HBASE-19715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-19715. --- Resolution: Cannot Reproduce Not on flaky lists. > Fix timing out test TestMultiRespectsLimits > --- > > Key: HBASE-19715 > URL: https://issues.apache.org/jira/browse/HBASE-19715 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Major > Attachments: HBASE-19715.test.patch, HBASE-19715.test.v2.patch, > failued.txt, passed.txt, screenshot-1.png, screenshot-2.png, > screenshot-3.png, screenshot-4.png, screenshot-5.png, screenshot-6.png > > > !screenshot-1.png|width=800px! > Attached logs for both cases, when it passes and fails. > Link (temporary) to logs: > passed: > http://104.198.223.121:8080/job/HBase-Flaky-Tests/33449/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestMultiRespectsLimits-output.txt/*view*/ > failed: > http://104.198.223.121:8080/job/HBase-Flaky-Tests/33455/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestMultiRespectsLimits-output.txt/*view*/ > Correlating across more runs, whenever the tests passes, it does so within > 10-30sec of 3min deadline for medium tests. > So i think we can make it pass by just increasing the timeout. > But I'm a bit skeptical after seeing all those long GC pauses (10sec +) in > the log. Test code doesn't seem to be doing anything that intensive. Are we > mismanaging the memory somewhere? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20030) Fix and reenable flakies TestQuotaThrottle and TestReplicasClient#testCancelOfMultiGet
[ https://issues.apache.org/jira/browse/HBASE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-20030: - Assignee: Peter Somogyi > Fix and reenable flakies TestQuotaThrottle and > TestReplicasClient#testCancelOfMultiGet > -- > > Key: HBASE-20030 > URL: https://issues.apache.org/jira/browse/HBASE-20030 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Peter Somogyi >Priority: Major > > Follow-on from HBASE-20029 to fix and reenable tests disabled so can cut a > beta-2 for hbase2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20967) TestFromClientSide3 fails with NPE
[
https://issues.apache.org/jira/browse/HBASE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi reassigned HBASE-20967:
-
Assignee: Peter Somogyi
> TestFromClientSide3 fails with NPE
> --
>
> Key: HBASE-20967
> URL: https://issues.apache.org/jira/browse/HBASE-20967
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: Peter Somogyi
>Priority: Major
> Attachments:
> org.apache.hadoop.hbase.client.TestFromClientSide3-output.txt
>
>
> https://builds.apache.org/job/HBASE-Flaky-Tests/35375/testReport/junit/org.apache.hadoop.hbase.client/TestFromClientSide3/testLockLeakWithDelta/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.TestFromClientSide3.find(TestFromClientSide3.java:995)
> at
> org.apache.hadoop.hbase.client.TestFromClientSide3.find(TestFromClientSide3.java:1002)
> at
> org.apache.hadoop.hbase.client.TestFromClientSide3.testLockLeakWithDelta(TestFromClientSide3.java:783)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20967) TestFromClientSide3 fails with NPE
[
https://issues.apache.org/jira/browse/HBASE-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-20967.
---
Resolution: Cannot Reproduce
> TestFromClientSide3 fails with NPE
> --
>
> Key: HBASE-20967
> URL: https://issues.apache.org/jira/browse/HBASE-20967
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: Peter Somogyi
>Priority: Major
> Attachments:
> org.apache.hadoop.hbase.client.TestFromClientSide3-output.txt
>
>
> https://builds.apache.org/job/HBASE-Flaky-Tests/35375/testReport/junit/org.apache.hadoop.hbase.client/TestFromClientSide3/testLockLeakWithDelta/
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.TestFromClientSide3.find(TestFromClientSide3.java:995)
> at
> org.apache.hadoop.hbase.client.TestFromClientSide3.find(TestFromClientSide3.java:1002)
> at
> org.apache.hadoop.hbase.client.TestFromClientSide3.testLockLeakWithDelta(TestFromClientSide3.java:783)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20754) quickstart guide should instruct folks to set JAVA_HOME to a JDK installation.
[
https://issues.apache.org/jira/browse/HBASE-20754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778096#comment-16778096
]
Sean Busbey commented on HBASE-20754:
-
yep, still an issue. I've added you as a contributor to jira, so you ought to
be able to assign this to yourself now if you'd like.
here's the general guide to contributing to documentation. if you need a
pointer on something let me know.
http://hbase.apache.org/book.html#appendix_contributing_to_documentation
> quickstart guide should instruct folks to set JAVA_HOME to a JDK installation.
> --
>
> Key: HBASE-20754
> URL: https://issues.apache.org/jira/browse/HBASE-20754
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Sean Busbey
>Priority: Major
> Labels: beginner
>
> The quickstart guide currently instructs folks to set JAVA_HOME, but to the
> wrong place
> {code}
> The JAVA_HOME variable should be set to a directory which contains the
> executable file bin/java. Most modern Linux operating systems provide a
> mechanism, such as /usr/bin/alternatives on RHEL or CentOS, for transparently
> switching between versions of executables such as Java. In this case, you can
> set JAVA_HOME to the directory containing the symbolic link to bin/java,
> which is usually /usr.
> JAVA_HOME=/usr
> {code}
> instead, it should tell folks to point it to a jdk installation and help them
> on how to find that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21820: --- Attachment: HBASE-21820.master.007.patch > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch, HBASE-21820.master.007.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21960) AuthFilter not configured for REST Jetty server
Josh Elser created HBASE-21960: -- Summary: AuthFilter not configured for REST Jetty server Key: HBASE-21960 URL: https://issues.apache.org/jira/browse/HBASE-21960 Project: HBase Issue Type: Bug Components: REST Reporter: Josh Elser Assignee: Josh Elser Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the AuthFilter in the filterchain for Jetty. Make sure we configure that class. I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) AuthFilter not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778145#comment-16778145 ] Josh Elser commented on HBASE-21960: Oops, typo. We miss RESTServletContainer, not AuthFilter. (AuthFilter was the part of the Pair we do configure :)) > AuthFilter not configured for REST Jetty server > --- > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > AuthFilter in the filterchain for Jetty. Make sure we configure that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Description: The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the RESTServletContainer in the filterchain for Jetty. Make sure we configure that class. I also have some testing to prevent a regression again in the future. was: The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the AuthFilter in the filterchain for Jetty. Make sure we configure that class. I also have some testing to prevent a regression again in the future. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer in the filterchain for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Summary: RESTServletContainer not configured for REST Jetty server (was: AuthFilter not configured for REST Jetty server) > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > AuthFilter in the filterchain for Jetty. Make sure we configure that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Status: Patch Available (was: Open) .001 here's a patch to fix the issue. The required change to fix the issue is quite small, but lots of tweaking is required to get RESTServer functional for kerberos-authn unit tests. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer in the filterchain for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Attachment: HBASE-21960.001.patch > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer in the filterchain for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Description: The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the RESTServletContainer as the base servlet for Jetty. Make sure we configure that class. I also have some testing to prevent a regression again in the future. was: The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the RESTServletContainer in the filterchain for Jetty. Make sure we configure that class. I also have some testing to prevent a regression again in the future. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[
https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778186#comment-16778186
]
Hadoop QA commented on HBASE-21960:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 3m
59s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
30s{color} | {color:red} hbase-rest in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 30s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-rest: The patch generated 12 new + 31 unchanged
- 9 fixed = 43 total (was 40) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
33s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
10s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
21s{color} | {color:red} hbase-rest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 23s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960210/HBASE-21960.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile findbugs hbaseanti checkstyle |
| uname | Linux 36ca420ec00f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c33ceb23d3 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| mvninstall |
https://builds.apache.org/job/Pre
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Attachment: HBASE-21960.002.patch > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778192#comment-16778192 ] Josh Elser commented on HBASE-21960: .002 fixes a missed import. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[
https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778235#comment-16778235
]
Hadoop QA commented on HBASE-21960:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} hbase-rest: The patch generated 12 new + 31 unchanged
- 9 fixed = 43 total (was 40) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 39s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
52s{color} | {color:red} hbase-rest generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 36s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-rest |
| | org.apache.hadoop.hbase.rest.RESTServer.conf isn't final and can't be
protected from malicious code At RESTServer.java:be protected from malicious
code At RESTServer.java:[line 105] |
| | org.apache.hadoop.hbase.rest.RESTServer.SKIP_LOGIN_KEY isn't final but
should be At RESTServer.java:be At RESTServer.java:[line 95] |
| | Write to static field org.apache.hadoop.hbase.rest.RESTServer.conf from
instance method new org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:from instance method new
org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:[line 111] |
| Failed junit tests | hadoop.hbase.rest.TestSecureRESTServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960215/HBASE-21960.002.patch
|
|
[jira] [Updated] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21935: -- Attachment: HBASE-21935.branch-2.1.002.patch > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20724) Sometimes some compacted storefiles are still opened after region failover
[
https://issues.apache.org/jira/browse/HBASE-20724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778441#comment-16778441
]
Hadoop QA commented on HBASE-20724:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
19s{color} | {color:blue} hbase-server in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 59s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
5s{color} | {color:red} hbase-server: The patch generated 1 new + 63 unchanged
- 3 fixed = 64 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 32s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
27s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}254m 42s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}314m 13s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestSplitWALManager |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.master.procedure.TestServerCrashProcedureWithReplicas |
| | hadoop.hbas
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778456#comment-16778456 ] stack commented on HBASE-21935: --- v2 addresses nice review by [~Apache9] and [~psomogyi] up on rb. Will exercise the script doing the 2.0.5 RC. Waiting on a commit before starting it up. Will update the patch with the fruit of this RM'ing exercise. > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778448#comment-16778448 ] Josh Elser commented on HBASE-21960: I have a first for the first issue, but that uncovers a second where the proxyuser configuration isn't taking effect in the unit test. This passes on my laptop, but it does fail similarly on a linux box that I have. I'm guessing QA will also run into that second issue, so trying to get it passing on my linux box before putting up a third patch. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21666) Break up the TestExportSnapshot UTs; they can timeout
[
https://issues.apache.org/jira/browse/HBASE-21666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778462#comment-16778462
]
Tak Lon (Stephen) Wu commented on HBASE-21666:
--
I have done investigation below, and I found the hanging/slow is related to
test node's network setup and local disk issue. I'd like to propose the
solution to be fail fast instead of timeout at 780+ when possible.
First of all, test methods in {{TestExportSnapshot}} contains two phases of
operations, operations in Mini HBase Cluster and operations in Mini MR Cluster,
and we are only snapshotting 50 rows into a test table (the data is very small).
So, the timeout issue is related the followings
1. the building node has an `incorrect` network interface setup such that
a. it hangs the HDFS file operations e.g.
{quote}2019-02-25 22:28:36,099 ERROR [ClientFinalizer-shutdown-hook]
hdfs.DFSClient(949): Failed to close inode 16420
java.io.EOFException: End of File Exception between local host is:
"f45c89a57f29.ant.amazon.com/192.168.1.15"; destination host is:
"localhost":54524; : java.io.EOFException; For more details see:
[http://wiki.apache.org/hadoop/EOFException]
{quote}
b. server (region server or hmaster) cannot be connected or regions cannot
be assigned and kept retrying till timeout, e.g.
{quote}2019-02-26 09:27:54,754 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=57922]
client.RpcRetryingCallerImpl(132): Call exception, tries=10, retries=19,
started=96205 ms ago, cancelled=false, msg=Call to
f45c89a57f29-2.local/10.63.166.57:57926 failed on local exception:
org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed
servers list: f45c89a57f29-2.local/10.63.166.57:57926, details=row
'testtb-testExportFileSystemStateWithSkipTmp' on table 'hbase:meta' at
region=hbase:meta,,1.1588230740,
hostname=f45c89a57f29-2.local,57926,1551201763075, seqNum=-1, see
[https://s.apache.org/timeout],
exception=org.apache.hadoop.hbase.ipc.FailedServerException: Call to
f45c89a57f29-2.local/10.63.166.57:57926 failed on local exception:
org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed
servers list: f45c89a57f29-2.local/10.63.166.57:57926`
{quote}
2. the building node has an out of disk space issue such node manager is not in
the health state, e.g. I saw from the node manger UI {{1/1 local-dirs are bad:
/yarn/nm; 1/1 log-dirs are bad: /yarn/container-logs}} even if we have set
{{yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage}}
to 99%
In above cases, assuming case 1) is an node setup issues (e.g. in
{{/etc/hosts}}) that can be fixed by the infra admin or the contributor who is
running the unit test on their laptop/machine, we don't need to fix it.
for case 2), I'm thinking to set a new value
{{yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb}} to 128MB
(should be enough for log-dirs and local-dirs) to fail fast when starting the
miniMRCluster by
{{[TestExportSnapshot#setUpBeforeClass|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java#L100-L104]}}
instead of timeout for 780+ seconds
In fact, if the building node does not have any of the connections and disk
issues, the average time of running all tests within {{TestExportSnapshot}} is
about 280 seconds and IMO it won't be able to speedup with splitting some of
the test methods into a separate classes and tests of each class are executed
in a sequential order (are we running tests in parallel especially for
{{TestExportSnapshot}} which labeled as {{LargeTests}}? when I tested with
{{mvn test -PrunAllTests -Dtest=TestExportSnapshot}}, I didn't see methods are
running concurrently even if I found the {{surefire.secondPartForkCount=5}} for
{{runAllTests}}).
So, if we think disk space issue of YARN's nodemanager should be failed fast
when running tests, proposed code change in
{{HBaseTestingUtility#startMiniMapReduceCluster}} should be as below.
Any comments?
{code:java}
@@ -2736,6 +2736,8 @@ public class HBaseTestingUtility extends
HBaseZKTestingUtility {
conf.setIfUnset(
"yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage",
"99.0");
+// Make sure we have enough disk space for log-dirs and local-dirs
+
conf.setIfUnset("yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb",
"128");
startMiniMapReduceCluster(2);
return mrCluster;
}
{code}
> Break up the TestExportSnapshot UTs; they can timeout
> -
>
> Key: HBASE-21666
> URL: https://issues.apache.org/jira/browse/HBASE-21666
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assi
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778451#comment-16778451 ] stack commented on HBASE-21960: --- Took a look. All seems fine. Looks like the hard-won issue of a bunch of trial-and-error. +1 after all tests pass. Nice test. Is this intentional? log4j.logger.org.apache.directory=WARN > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21666) Break up the TestExportSnapshot UTs; they can timeout
[
https://issues.apache.org/jira/browse/HBASE-21666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778462#comment-16778462
]
Tak Lon (Stephen) Wu edited comment on HBASE-21666 at 2/26/19 6:54 PM:
---
I have done investigation below, and I found the hanging/slow is related to
test node's network setup and local disk issue. I'd like to propose the
solution to be fail fast instead of timeout at 780+ when possible.
First of all, test methods in {{TestExportSnapshot}} contains two phases of
operations, operations in Mini HBase Cluster and operations in Mini MR Cluster,
and we are only snapshotting 50 rows into a test table (the data is very small).
So, the timeout issue is related the followings
1. the building node has an `incorrect` network interface setup such that
a. it hangs the HDFS file operations e.g.
{quote}2019-02-25 22:28:36,099 ERROR [ClientFinalizer-shutdown-hook]
hdfs.DFSClient(949): Failed to close inode 16420
java.io.EOFException: End of File Exception between local host is:
"f45c89a57f29.ant.amazon.com/192.168.1.15"; destination host is:
"localhost":54524; : java.io.EOFException; For more details see:
[http://wiki.apache.org/hadoop/EOFException]
{quote}
b. server (region server or hmaster) cannot be connected or regions cannot
be assigned and kept retrying till timeout, e.g.
{quote}2019-02-26 09:27:54,754 DEBUG
[RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=57922]
client.RpcRetryingCallerImpl(132): Call exception, tries=10, retries=19,
started=96205 ms ago, cancelled=false, msg=Call to
f45c89a57f29-2.local/10.63.166.57:57926 failed on local exception:
org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed
servers list: f45c89a57f29-2.local/10.63.166.57:57926, details=row
'testtb-testExportFileSystemStateWithSkipTmp' on table 'hbase:meta' at
region=hbase:meta,,1.1588230740,
hostname=f45c89a57f29-2.local,57926,1551201763075, seqNum=-1, see
[https://s.apache.org/timeout],
exception=org.apache.hadoop.hbase.ipc.FailedServerException: Call to
f45c89a57f29-2.local/10.63.166.57:57926 failed on local exception:
org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed
servers list: f45c89a57f29-2.local/10.63.166.57:57926`
{quote}
2. the building node has an out of disk space issue such node manager is not in
the health state, e.g. I saw from the node manger UI {{1/1 local-dirs are bad:
/yarn/nm; 1/1 log-dirs are bad: /yarn/container-logs}} even if we have set
{{yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage}}
to 99%
In above cases, assuming case 1) is an node setup issues (e.g. in
{{/etc/hosts}}) that can be fixed by the infra admin or the contributor who is
running the unit test on their laptop/machine, we don't need to fix it.
for case 2), I'm thinking to set a new value
{{yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb}} to 128MB
(should be enough for log-dirs and local-dirs) to fail fast when starting the
miniMRCluster by
{{[TestExportSnapshot#setUpBeforeClass|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java#L100-L104]}}
instead of timeout for 780+ seconds
In fact, if the building node does not have any of the connections and disk
issues, the average time of running all (7) tests within {{TestExportSnapshot}}
is about 280 seconds and IMO it won't be able to speedup with splitting some of
the test methods into a separate classes and tests of each class are executed
in a sequential order (are we running tests in parallel especially for
{{TestExportSnapshot}} which labeled as {{LargeTests}}? when I tested with
{{mvn test -PrunAllTests -Dtest=TestExportSnapshot}}, I didn't see methods are
running concurrently even if I found the {{surefire.secondPartForkCount=5}} for
{{runAllTests}}, but if anyone confirm it does, we can also separate each
method in {{TestExportSnapshot}} to different classes).
So, if we think disk space issue of YARN's nodemanager should be failed fast
when running tests, proposed code change in
{{HBaseTestingUtility#startMiniMapReduceCluster}} should be as below.
Any comments?
{code:java}
@@ -2736,6 +2736,8 @@ public class HBaseTestingUtility extends
HBaseZKTestingUtility {
conf.setIfUnset(
"yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage",
"99.0");
+// Make sure we have enough disk space for log-dirs and local-dirs
+
conf.setIfUnset("yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb",
"128");
startMiniMapReduceCluster(2);
return mrCluster;
}
{code}
was (Author: taklwu):
I have done investigation below, and I found the hanging/slow is related to
test node's network setup and local disk issue. I'd like to propose the
solution to be
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[
https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778470#comment-16778470
]
Josh Elser commented on HBASE-21960:
{quote}Is this intentional?
{quote}
Yeah, it's silencing all of the stuff sitting behind MiniKDC/Kerby. We care
about it 0%
.003 incoming shortly. Something is different on a real linux box with Hadoop
user lookups. "hard-coding" everythign to be one group fixes it and test passes
for me.
> RESTServletContainer not configured for REST Jetty server
> -
>
> Key: HBASE-21960
> URL: https://issues.apache.org/jira/browse/HBASE-21960
> Project: HBase
> Issue Type: Bug
> Components: REST
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4
>
> Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch
>
>
> The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the
> RESTServletContainer as the base servlet for Jetty. Make sure we configure
> that class.
> I also have some testing to prevent a regression again in the future.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21820) Implement CLUSTER quota scope
[
https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778481#comment-16778481
]
Hadoop QA commented on HBASE-21820:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
34s{color} | {color:blue} hbase-server in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} hbase-server: The patch generated 0 new + 11
unchanged - 1 fixed = 11 total (was 12) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch passed checkstyle in hbase-shell {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
11s{color} | {color:red} The patch generated 16 new + 133 unchanged - 9 fixed =
149 total (was 142) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 11s{color} | {color:orange} The patch generated 46 new + 433 unchanged - 0
fixed = 479 total (was 433) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2669 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 1m
26s{color} | {color:red} The patch 300 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
5s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}132m
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Attachment: HBASE-21960.003.patch > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch, > HBASE-21960.003.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778522#comment-16778522 ] Sean Busbey commented on HBASE-21935: - TODO: update nightly check of RM steps to use this on relevant branches? > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778523#comment-16778523 ] Sean Busbey commented on HBASE-21935: - > TODO: Add check that all issues resolved in JIRA before starting? I haven't been tracking how y'all are doing this in branches-2 but in 1.2 land I usually have a "Release 1.2.xx" jira that I use for branch fixups. It's usually open until I have finished all the release steps. How do y'all track that info? > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778528#comment-16778528 ] stack commented on HBASE-21935: --- Thanks for input [~busbey] What is nightly doing to help out RMing? I think us on branch-2 do similar to your practice making a JIRA and then sub-jiras to do each substep (should be less of this w/ this script in place -- TODO: write it up). > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778540#comment-16778540 ] Sean Busbey commented on HBASE-21935: - bq. I think us on branch-2 do similar to your practice making a JIRA and then sub-jiras to do each substep (should be less of this w/ this script in place – TODO: write it up). This means there'll be an open JIRA with the relevant fix version, no? bq. What is nightly doing to help out RMing? Some time ago I set up a nightly test that followed the steps in the book, because one time when I went to try to make an RC the steps failed. (I don't remember specifics of the failure any more). here is the starts of the test definition: https://github.com/apache/hbase/blob/master/dev-support/Jenkinsfile#L495 I only really pay attention to how it's doing for branch-1.2, so I have no idea if it's currently useful for branches-2. > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778540#comment-16778540 ] Sean Busbey edited comment on HBASE-21935 at 2/26/19 8:15 PM: -- bq. I think us on branch-2 do similar to your practice making a JIRA and then sub-jiras to do each substep (should be less of this w/ this script in place – TODO: write it up). This means there'll be an open JIRA with the relevant fix version, no? bq. What is nightly doing to help out RMing? Some time ago I set up a nightly test that followed the steps in the book, because one time when I went to try to make an RC the steps failed. (I don't remember specifics of the failure any more). here is the start of the test definition on current master: [dev-support/Jenkinsfile line 495|https://github.com/apache/hbase/blob/7377fcd29bf45208214973547facf4853620fba8/dev-support/Jenkinsfile#L495] I only really pay attention to how it's doing for branch-1.2, so I have no idea if it's currently useful for branches-2. was (Author: busbey): bq. I think us on branch-2 do similar to your practice making a JIRA and then sub-jiras to do each substep (should be less of this w/ this script in place – TODO: write it up). This means there'll be an open JIRA with the relevant fix version, no? bq. What is nightly doing to help out RMing? Some time ago I set up a nightly test that followed the steps in the book, because one time when I went to try to make an RC the steps failed. (I don't remember specifics of the failure any more). here is the starts of the test definition: https://github.com/apache/hbase/blob/master/dev-support/Jenkinsfile#L495 I only really pay attention to how it's doing for branch-1.2, so I have no idea if it's currently useful for branches-2. > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[
https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778549#comment-16778549
]
Hadoop QA commented on HBASE-21960:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} hbase-rest: The patch generated 4 new + 31 unchanged -
9 fixed = 35 total (was 40) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 9s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red} hbase-rest generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 5s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-rest |
| | org.apache.hadoop.hbase.rest.RESTServer.conf isn't final and can't be
protected from malicious code At RESTServer.java:be protected from malicious
code At RESTServer.java:[line 106] |
| | Write to static field org.apache.hadoop.hbase.rest.RESTServer.conf from
instance method new org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:from instance method new
org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:[line 112] |
| Failed junit tests | hadoop.hbase.rest.TestSecureRESTServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960222/HBASE-21960.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile findbugs hbaseanti checkstyle |
| uname
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778558#comment-16778558 ] stack commented on HBASE-21935: --- bq. This means there'll be an open JIRA with the relevant fix version, no? Yeah. Thats how its been done in past. Need to carry on in this manner or suggest something different bq. Some time ago I set up a nightly test that followed the steps in the book I see. Yeah, would be good to slot this script in if we can. Will look into it. > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Attachment: HBASE-21960.004.patch > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch, > HBASE-21960.003.patch, HBASE-21960.004.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778571#comment-16778571 ] Josh Elser commented on HBASE-21960: .004 At a loss again. Trying to just do proxyuser.user configuration instead of proxyuser.groups configuration. Still passes on both of my boxes. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch, > HBASE-21960.003.patch, HBASE-21960.004.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[
https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778596#comment-16778596
]
Hadoop QA commented on HBASE-21960:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} hbase-rest: The patch generated 9 new + 31 unchanged -
9 fixed = 40 total (was 40) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 59s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red} hbase-rest generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 41s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-rest |
| | org.apache.hadoop.hbase.rest.RESTServer.conf isn't final and can't be
protected from malicious code At RESTServer.java:be protected from malicious
code At RESTServer.java:[line 106] |
| | Write to static field org.apache.hadoop.hbase.rest.RESTServer.conf from
instance method new org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:from instance method new
org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:[line 112] |
| Failed junit tests | hadoop.hbase.rest.TestSchemaResource |
| | hadoop.hbase.rest.TestMultiRowResource |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960231/HBASE-21960.004.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778630#comment-16778630 ] Josh Elser commented on HBASE-21960: Whew. Over the hurdle now. Looks like these might be valid regressions on my later iterations. Fixing. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch, > HBASE-21960.003.patch, HBASE-21960.004.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21947) TestShell is broken after we remove the jackson dependencies
[
https://issues.apache.org/jira/browse/HBASE-21947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778645#comment-16778645
]
Hudson commented on HBASE-21947:
Results for branch branch-2
[build #1713 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1713/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1713//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1713//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1713//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1713//console].
> TestShell is broken after we remove the jackson dependencies
>
>
> Key: HBASE-21947
> URL: https://issues.apache.org/jira/browse/HBASE-21947
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21947-v1.patch, HBASE-21947.patch
>
>
> {noformat}
> Error: test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
> NameError: cannot load Java class com.fasterxml.jackson.databind.ObjectMapper
> org/jruby/javasupport/JavaClass.java:286:in `for_name'
> org/jruby/javasupport/JavaUtilities.java:34:in `get_proxy_class'
> uri:classloader:/jruby/java/core_ext/object.rb:49:in `block in java_import'
> org/jruby/RubyArray.java:2486:in `map'
> uri:classloader:/jruby/java/core_ext/object.rb:36:in `java_import'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:77:in
> `tasksOnHost'
> src/test/ruby/hbase/taskmonitor_test.rb:33:in `block in
> test_tasksOnHost_should_return_tasks_list'
> 30: filter = 'all'
> 31: hosts = admin.getRegionServers()
> 32: hosts.each do |host|
> => 33: tasks = taskmonitor.tasksOnHost(filter,host)
> 34: assert(tasks.length > 0)
> 35: end
> 36: end
> org/jruby/RubyArray.java:1734:in `each'
> src/test/ruby/hbase/taskmonitor_test.rb:32:in `block in
> test_tasksOnHost_should_return_tasks_list'
> {noformat}
> Seems we are using jackson in hbase-shell.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21947) TestShell is broken after we remove the jackson dependencies
[
https://issues.apache.org/jira/browse/HBASE-21947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778689#comment-16778689
]
Hudson commented on HBASE-21947:
Results for branch branch-2.2
[build #69 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/69/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/69//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/69//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/69//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/69//console].
> TestShell is broken after we remove the jackson dependencies
>
>
> Key: HBASE-21947
> URL: https://issues.apache.org/jira/browse/HBASE-21947
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21947-v1.patch, HBASE-21947.patch
>
>
> {noformat}
> Error: test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
> NameError: cannot load Java class com.fasterxml.jackson.databind.ObjectMapper
> org/jruby/javasupport/JavaClass.java:286:in `for_name'
> org/jruby/javasupport/JavaUtilities.java:34:in `get_proxy_class'
> uri:classloader:/jruby/java/core_ext/object.rb:49:in `block in java_import'
> org/jruby/RubyArray.java:2486:in `map'
> uri:classloader:/jruby/java/core_ext/object.rb:36:in `java_import'
> /home/jenkins/jenkins-slave/workspace/HBase-Flaky-Tests_master-M6IJIUBEFGBLJA2QZFJ5NFMN4UCZCOJZHSKUZTUZSLWEH6TJXWBQ/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:77:in
> `tasksOnHost'
> src/test/ruby/hbase/taskmonitor_test.rb:33:in `block in
> test_tasksOnHost_should_return_tasks_list'
> 30: filter = 'all'
> 31: hosts = admin.getRegionServers()
> 32: hosts.each do |host|
> => 33: tasks = taskmonitor.tasksOnHost(filter,host)
> 34: assert(tasks.length > 0)
> 35: end
> 36: end
> org/jruby/RubyArray.java:1734:in `each'
> src/test/ruby/hbase/taskmonitor_test.rb:32:in `block in
> test_tasksOnHost_should_return_tasks_list'
> {noformat}
> Seems we are using jackson in hbase-shell.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21082) Reimplement assign/unassign related procedure metrics
[ https://issues.apache.org/jira/browse/HBASE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778743#comment-16778743 ] Duo Zhang commented on HBASE-21082: --- I think this should be done before we release 2.2.0? [~zghaobac]. As the TRSP stuffs will be shipped in 2.2.0. > Reimplement assign/unassign related procedure metrics > - > > Key: HBASE-21082 > URL: https://issues.apache.org/jira/browse/HBASE-21082 > Project: HBase > Issue Type: Sub-task > Components: amv2, metrics >Reporter: Duo Zhang >Priority: Critical > Fix For: 3.0.0, 2.3.0 > > > As after HBASE-20881, we only have one TRSP procedure to handle all the > assign/unassign/move/reopen operations, we need to reimplement(redefine?) the > metrics for these procedures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21820: --- Attachment: HBASE-21820.master.007.patch > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch, HBASE-21820.master.007.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21820: --- Attachment: (was: HBASE-21820.master.007.patch) > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21960: --- Attachment: HBASE-21960.005.patch > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch, > HBASE-21960.003.patch, HBASE-21960.004.patch, HBASE-21960.005.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[ https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778784#comment-16778784 ] Josh Elser commented on HBASE-21960: .005 Inadvertently dropped some important config changes for CSRF protections which were chucked into the RESTServer testing utility. > RESTServletContainer not configured for REST Jetty server > - > > Key: HBASE-21960 > URL: https://issues.apache.org/jira/browse/HBASE-21960 > Project: HBase > Issue Type: Bug > Components: REST >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.1.4 > > Attachments: HBASE-21960.001.patch, HBASE-21960.002.patch, > HBASE-21960.003.patch, HBASE-21960.004.patch, HBASE-21960.005.patch > > > The upgrade to Jetty 9 (HBASE-12894) appears to have missed including the > RESTServletContainer as the base servlet for Jetty. Make sure we configure > that class. > I also have some testing to prevent a regression again in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778785#comment-16778785 ] Duo Zhang commented on HBASE-21895: --- OK, seems the output for the error prone is not stable enough? I tried on master, there is no warnings for HMasterCommandLine, but the String.split is still there... And on the error prone guideline, http://errorprone.info/docs/installation It says that for java8 you need to add a bootstrap classpath and use a customized javac, but for us, we added a plexus-compiler-javac-errorprone dependency. Not sure if this is the problem? > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21820: --- Attachment: (was: HBASE-21820.master.007.patch) > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21906) Backport the CallQueueTooBigException related changes in HBASE-21875 to branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778788#comment-16778788 ] Duo Zhang commented on HBASE-21906: --- Missed this one... > Backport the CallQueueTooBigException related changes in HBASE-21875 to > branch-2.1 > -- > > Key: HBASE-21906 > URL: https://issues.apache.org/jira/browse/HBASE-21906 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.1.4 > > Attachments: HBASE-21906-branch-2.1.patch, > HBASE-21906-branch-2.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21820) Implement CLUSTER quota scope
[ https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21820: --- Attachment: HBASE-21820.master.007.patch > Implement CLUSTER quota scope > - > > Key: HBASE-21820 > URL: https://issues.apache.org/jira/browse/HBASE-21820 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-21820.master.001.patch, > HBASE-21820.master.002.patch, HBASE-21820.master.003.patch, > HBASE-21820.master.004.patch, HBASE-21820.master.005.patch, > HBASE-21820.master.006.patch, HBASE-21820.master.007.patch > > > There are two kinds of quota scope: CLUSTER and MACHINE. CLUSTER quota means > quota limit is shared by all machines of cluster. MACHINE quota means quota > limit is used by single region server. > Currently, all set quota operations use MACHINE scope as default and CLUSTER > scope has not been implemented. So open this issue to implement CLUSTER quota > scope. > To split cluster quota limit to machines, the basic idea is for user, > namespace, user over namespace quota, use [ClusterLimit / RSNum] as machine > limit. For table and user over table quota, use [ClusterLimit / > TotalTableRegionNum * MachineTableRegionNum] as machine limit. Suggestions > are welcomed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21906) Backport the CallQueueTooBigException related changes in HBASE-21875 to branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21906: -- Attachment: HBASE-21906-branch-2.1.patch > Backport the CallQueueTooBigException related changes in HBASE-21875 to > branch-2.1 > -- > > Key: HBASE-21906 > URL: https://issues.apache.org/jira/browse/HBASE-21906 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.1.4 > > Attachments: HBASE-21906-branch-2.1.patch, > HBASE-21906-branch-2.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21961) Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we tried to locate before a non empty row
Duo Zhang created HBASE-21961: - Summary: Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we tried to locate before a non empty row Key: HBASE-21961 URL: https://issues.apache.org/jira/browse/HBASE-21961 Project: HBase Issue Type: Bug Components: asyncclient, Client Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 Find this when implementing HBASE-21717, which means it is good that our UTs for sync client really covers a lot of corner cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21962) Filters do not work in ThriftTable
Allan Yang created HBASE-21962: -- Summary: Filters do not work in ThriftTable Key: HBASE-21962 URL: https://issues.apache.org/jira/browse/HBASE-21962 Project: HBase Issue Type: Sub-task Components: Thrift Reporter: Allan Yang Assignee: Allan Yang Fix For: 3.0.0, 2.2.0 Filters in ThriftTable is not working, this issue is to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21820) Implement CLUSTER quota scope
[
https://issues.apache.org/jira/browse/HBASE-21820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778807#comment-16778807
]
Hadoop QA commented on HBASE-21820:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
18s{color} | {color:blue} hbase-server in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m
27s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 2m
58s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 58s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
3s{color} | {color:green} root: The patch generated 0 new + 12 unchanged - 1
fixed = 12 total (was 13) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
10s{color} | {color:red} The patch generated 16 new + 133 unchanged - 9 fixed =
149 total (was 142) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 10s{color} | {color:orange} The patch generated 46 new + 433 unchanged - 0
fixed = 479 total (was 433) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
14s{color} | {color:red} patch has 396 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
45s{color} | {color:red} The patch causes 396 errors with Hadoop v2.7.4.
{color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
34s{color} | {color:red} The patch causes 396 errors with Hadoop v3.0.0.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
35s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 12m 15s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/
[jira] [Commented] (HBASE-21906) Backport the CallQueueTooBigException related changes in HBASE-21875 to branch-2.1
[
https://issues.apache.org/jira/browse/HBASE-21906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778802#comment-16778802
]
Hadoop QA commented on HBASE-21906:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 4m
20s{color} | {color:red} root in branch-2.1 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 2m
32s{color} | {color:red} hbase-server in branch-2.1 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
43s{color} | {color:green} branch-2.1 passed {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 4m
48s{color} | {color:red} branch has 20 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
44s{color} | {color:red} hbase-server in branch-2.1 failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m
36s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
36s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 36s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
11s{color} | {color:red} patch has 20 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
56s{color} | {color:red} The patch causes 20 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
51s{color} | {color:red} The patch causes 20 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
32s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21906 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960270/HBASE-21906-branch-2.1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3e02d0252def 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.1 / dcc22d3e41 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HBASE-Build/16138/artifact/patchprocess/branch-mvninstall-root.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/16138/artifact/patchprocess/branch-compi
[jira] [Commented] (HBASE-21960) RESTServletContainer not configured for REST Jetty server
[
https://issues.apache.org/jira/browse/HBASE-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778813#comment-16778813
]
Hadoop QA commented on HBASE-21960:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} hbase-rest: The patch generated 9 new + 33 unchanged -
9 fixed = 42 total (was 42) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
55s{color} | {color:red} hbase-rest generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
10s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-rest |
| | org.apache.hadoop.hbase.rest.RESTServer.conf isn't final and can't be
protected from malicious code At RESTServer.java:be protected from malicious
code At RESTServer.java:[line 106] |
| | Write to static field org.apache.hadoop.hbase.rest.RESTServer.conf from
instance method new org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:from instance method new
org.apache.hadoop.hbase.rest.RESTServer(Configuration) At
RESTServer.java:[line 112] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960268/HBASE-21960.005.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile findbugs hbaseanti checkstyle |
| uname | Linux 8a3e96228f00 4.4.0-138-generic #164~14.04.1-Ubun
[jira] [Updated] (HBASE-21916) Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool
[ https://issues.apache.org/jira/browse/HBASE-21916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21916: - Attachment: HBASE-21916.HBASE-21879.v5.patch > Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool > --- > > Key: HBASE-21916 > URL: https://issues.apache.org/jira/browse/HBASE-21916 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.1.4 > > Attachments: HBASE-21916.HBASE-21879.v1.patch, > HBASE-21916.HBASE-21879.v2.patch, HBASE-21916.HBASE-21879.v3.patch, > HBASE-21916.HBASE-21879.v3.patch, HBASE-21916.HBASE-21879.v4.patch, > HBASE-21916.HBASE-21879.v5.patch, HBASE-21916.v1.patch, HBASE-21916.v2.patch, > HBASE-21916.v3.patch, HBASE-21916.v4.patch, HBASE-21916.v5.patch > > > Now our read/write path allocate ByteBuffer from the ByteBufferPool, but we > need consider the minSizeForReservoirUse for better utilization, those > allocate/free api are some static methods, not so good to use. > For HBASE-21879, we need an universal ByteBuffer allocator to manage all the > ByteBuffers through the entire read path, so create this issue. > Will upload a patch to abstract an ByteBufAllocator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-21962) Filters do not work in ThriftTable
[ https://issues.apache.org/jira/browse/HBASE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-21962 started by Allan Yang. -- > Filters do not work in ThriftTable > -- > > Key: HBASE-21962 > URL: https://issues.apache.org/jira/browse/HBASE-21962 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Filters in ThriftTable is not working, this issue is to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21962) Filters do not work in ThriftTable
[ https://issues.apache.org/jira/browse/HBASE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-21962: --- Status: Patch Available (was: In Progress) > Filters do not work in ThriftTable > -- > > Key: HBASE-21962 > URL: https://issues.apache.org/jira/browse/HBASE-21962 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Filters in ThriftTable is not working, this issue is to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21962) Filters do not work in ThriftTable
[ https://issues.apache.org/jira/browse/HBASE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-21962: --- Attachment: HBASE-21962.patch > Filters do not work in ThriftTable > -- > > Key: HBASE-21962 > URL: https://issues.apache.org/jira/browse/HBASE-21962 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21962.patch > > > Filters in ThriftTable is not working, this issue is to fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21961) Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we tried to locate before a non empty row
[ https://issues.apache.org/jira/browse/HBASE-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21961: -- Attachment: HBASE-21961.patch > Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we > tried to locate before a non empty row > > > Key: HBASE-21961 > URL: https://issues.apache.org/jira/browse/HBASE-21961 > Project: HBase > Issue Type: Bug > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-21961.patch > > > Find this when implementing HBASE-21717, which means it is good that our UTs > for sync client really covers a lot of corner cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21961) Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we tried to locate before a non empty row
[ https://issues.apache.org/jira/browse/HBASE-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21961: -- Status: Patch Available (was: Open) > Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we > tried to locate before a non empty row > > > Key: HBASE-21961 > URL: https://issues.apache.org/jira/browse/HBASE-21961 > Project: HBase > Issue Type: Bug > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-21961.patch > > > Find this when implementing HBASE-21717, which means it is good that our UTs > for sync client really covers a lot of corner cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-21943) The usage of RegionLocations.mergeRegionLocations is wrong for async client
[ https://issues.apache.org/jira/browse/HBASE-21943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-21943: --- Fix 2.1 compile error. > The usage of RegionLocations.mergeRegionLocations is wrong for async client > --- > > Key: HBASE-21943 > URL: https://issues.apache.org/jira/browse/HBASE-21943 > Project: HBase > Issue Type: Bug > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-21943-UT-v1.patch, HBASE-21943-UT.patch, > HBASE-21943-UT.patch, HBASE-21943-v1.patch, HBASE-21943-v2.patch, > HBASE-21943.patch > > > In AsyncRegionLocatorHelper.mergeRegionLocations we create a new > RegionLocations and call mergeRegionLocations on it, expected that it will be > changed by this method, but actually the method will not modify the object > itself, it will return a new one... > And we are lucky that we create the RegionLocations with the new locations, > so usually we will get update result. But when testing HBASE-21717, we meet > another bug in AsyncNonMetaRegionLocator.isEqual, where we missed a '!' when > checking server name equals... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21906) Backport the CallQueueTooBigException related changes in HBASE-21875 to branch-2.1
[ https://issues.apache.org/jira/browse/HBASE-21906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21906: -- Attachment: HBASE-21906-branch-2.1.patch > Backport the CallQueueTooBigException related changes in HBASE-21875 to > branch-2.1 > -- > > Key: HBASE-21906 > URL: https://issues.apache.org/jira/browse/HBASE-21906 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.1.4 > > Attachments: HBASE-21906-branch-2.1.patch, > HBASE-21906-branch-2.1.patch, HBASE-21906-branch-2.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
