[jira] [Updated] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19935: --- Attachment: HBASE-19935.HBASE-19064.001.patch > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19935: --- Status: Patch Available (was: Open) > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19528) Major Compaction Tool
[
https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353535#comment-16353535
]
Hadoop QA commented on HBASE-19528:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 9m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 34s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m
48s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19528 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909365/HBASE-19528.v2.branch-1%20%281%29.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d270822807d3 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 11e3a83870 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11407/testReport/ |
| Max. process+thread count | 5266 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11407/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Major Compaction Tool
> --
>
> Key: HBASE-19528
> URL: https://issues.apache.org/j
[jira] [Commented] (HBASE-19937) Enable rsgroup NPE in CreateTableProcedure
[
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353537#comment-16353537
]
Xiaolin Ha commented on HBASE-19937:
When enable rsgroup, we should set config "hbase.master.loadbalancer.class" to
"org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer". When master
started, it will first load SystemCoprocessors by
{code:java}
844 his.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
and then initialize load balancer by
{code:java}
866 this.balancer.setMasterServices(this);
867 this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
868 this.balancer.initialize();
{code}
but because of HMaster using classes relevant of rsgroup just at the time of
load SystemCoprocessors,
the codes without fix create rsgroup system table at that time, too.
All system tables created by using CreateTableProcedure.executeFromState, which
will call getBalancer().roundRobinAssignment,
getBalancer() returns LoadBalancerFactory.getLoadBalancer(conf), which we has
set to be RSGroupBasedLoadBalancer.
RSGroupBasedLoadBalancer called initialize() after create hbase:rsgroup table,
so it will be throw NPE.
> Enable rsgroup NPE in CreateTableProcedure
> --
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: HBASE-19937.branch-2.001.patch
>
>
> When enable rsgroup, it may throws NPE as follows,
> 2018-02-02,16:12:45,688 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS;
> CreateTableProcedure table=hbase:rsgroup
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>
> As a result of CreateTableProcedure.rollbackState, it may then print logs
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> Failed to perform check
> java.io.IOException: Failed to create group table.
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>
> After some auto-retries, it loops running the thread RSGroupStartupWorker,
> will print logs as follows,
> 2018-02-02,16:23:17,626 INFO
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0,
> rootMetaFound=true
> 2018-02-02,16:23:17,730 INFO
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0,
> rootMetaFound=true
> 2018-02-02,16:23:17,834 INFO
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0,
> rootMetaFound=true
> 2018-02-02,16:23:17,937 INFO
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0,
> rootMetaFound=true
>
> And using shells of rsgroup, it will tips that currently is in "offline mode".
>
> The reason of this problem is that CreateTableProcedure used
> RSGroupBasedLoadBalancer, who has member variables initialized depending on
> return of Create
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353542#comment-16353542 ] Duo Zhang commented on HBASE-19929: --- OK, there is still a dead lock. We have a DrainBarrier that prevents the closing of a WAL if there are outoging flushes. Let me see how to deal with this. Thanks. > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19934) HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting
[
https://issues.apache.org/jira/browse/HBASE-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353550#comment-16353550
]
Hadoop QA commented on HBASE-19934:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-1 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
37s{color} | {color:red} The patch causes 44 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
29s{color} | {color:red} The patch causes 44 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m
58s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | HBASE-19934 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909367/HBASE-19934-branch-1.patch
|
| Optional Tests | asflicense
[jira] [Commented] (HBASE-19927) TestFullLogReconstruction flakey
[ https://issues.apache.org/jira/browse/HBASE-19927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353554#comment-16353554 ] Hudson commented on HBASE-19927: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4535 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4535/]) HBASE-19927 Addendum join on RegionServerThread instead of HRegionServer (zhangduo: rev f5197979aaac7e36b6af36b86ea8dc8d7774fabe) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestFullLogReconstruction.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java > TestFullLogReconstruction flakey > > > Key: HBASE-19927 > URL: https://issues.apache.org/jira/browse/HBASE-19927 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: stack >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19927-addendum.patch, HBASE-19927.patch, js, out > > > Fails pretty frequently in hadoopqa builds. > There is a recent hang in > org.apache.hadoop.hbase.TestFullLogReconstruction.tearDownAfterClass(TestFullLogReconstruction.java:68) > In here... > https://builds.apache.org/job/PreCommit-HBASE-Build/11363/testReport/org.apache.hadoop.hbase/TestFullLogReconstruction/org_apache_hadoop_hbase_TestFullLogReconstruction/ > ... see here. > Thread 1250 (RS_CLOSE_META-edd281aedb18:59863-0): > State: TIMED_WAITING > Blocked count: 92 > Waited count: 278 > Stack: > java.lang.Object.wait(Native Method) > > org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:133) > > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:718) > > org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:605) > > org.apache.hadoop.hbase.regionserver.wal.WALUtil.doFullAppendTransaction(WALUtil.java:154) > > org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeFlushMarker(WALUtil.java:81) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2645) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2356) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2328) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2319) > org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1531) > org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1437) > > org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:104) > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > java.lang.Thread.run(Thread.java:748) > We missed a signal? We need to do an interrupt? The log is not all there in > hadoopqa builds so hard to see all that is going on. This test is not in the > flakey set either -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19934) HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting
[
https://issues.apache.org/jira/browse/HBASE-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353556#comment-16353556
]
Hudson commented on HBASE-19934:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4535 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4535/])
HBASE-19934 HBaseSnapshotException when read replicas is enabled and (tedyu:
rev 11e3a8387006a5dd348fa91a336bce8d25455f4d)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClient.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClientWithRegionReplicas.java
> HBaseSnapshotException when read replicas is enabled and online snapshot is
> taken after region splitting
>
>
> Key: HBASE-19934
> URL: https://issues.apache.org/jira/browse/HBASE-19934
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19934-UT.patch, HBASE-19934-branch-1.patch,
> HBASE-19934-v2.patch, HBASE-19934-v3.patch, HBASE-19934-v3.patch,
> HBASE-19934.patch, HBASE-19934.patch, HBASE-19934.patch, HBASE-19934.patch
>
>
> Investigating HBASE-19893, I'm encountering another issue.
> Steps to reproduce are as follows:
> 1. Create a table
> {code:java}
> create "test", "cf", {REGION_REPLICATION => 2}{code}
> 2. Load data to the table
> {code:java}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}{code}
> 3. Split the table
> {code:java}
> split "test"{code}
> 4. Take a snapshot for the table
> {code:java}
> snapshot "test", "snap"{code}
> And I encountered the following error:
> {code:java}
> hbase(main):004:0> snapshot "test", "snap"
> ERROR: org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
> ss=snap table=test type=FLUSH } had an error. Procedure snap { waiting=[]
> done=[] }
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:379)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1144)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
> Failed taking snapshot { ss=snap table=test type=FLUSH } due to
> exception:Manifest region info {ENCODED => b910488a686644a7c1c85246d0d123d5,
> NAME => 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.',
> STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID =>
> 1}doesn't match expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a,
> NAME => 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY =>
> '', ENDKEY => '', OFFLINE => true, SPLIT =>
> true}:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Manifest
> region info {ENCODED => b910488a686644a7c1c85246d0d123d5, NAME =>
> 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 1}doesn't match
> expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a, NAME =>
> 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true}
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:82)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:306)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:368)
> ... 6 more
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
> Manifest region info {ENCODED => b910488a686644a7c1c85246d0d123d5, NAME =>
> 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 1}doesn't match
> expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a, NAME =>
> 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true}
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegionInfo(M
[jira] [Commented] (HBASE-19939) TestSplitTableRegionProcedure#testSplitWithoutPONR() and testRecoveryAndDoubleExecution() are failing with NPE
[
https://issues.apache.org/jira/browse/HBASE-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353555#comment-16353555
]
Hudson commented on HBASE-19939:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4535 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4535/])
HBASE-19939 Fixed NPE in tests (stack: rev
3bb8daa60565ec2f7955352e52c2f6379176d8c6)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
> TestSplitTableRegionProcedure#testSplitWithoutPONR() and
> testRecoveryAndDoubleExecution() are failing with NPE
> --
>
> Key: HBASE-19939
> URL: https://issues.apache.org/jira/browse/HBASE-19939
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0-beta-1
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: hbase-19939.master.001.patch,
> hbase-19939.master.002.patch
>
>
> Error is:
> {code:java}
> java.lang.AssertionError: found exception: java.lang.NullPointerException via
> CODE-BUG: Uncaught runtime exception: pid=154,
> state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS;
> SplitTableRegionProcedure table=testRecoveryAndDoubleExecution,
> parent=3d8d459ba395c2cf6b1e5c71aca92cfd,
> daughterA=c6531c10effa8e542159ab82a87bd75e,
> daughterB=ee34a9af88273b6c06e1a688fc50ed6e:java.lang.NullPointerException:
> at
> org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure.testRecoveryAndDoubleExecution(TestSplitTableRegionProcedure.java:411){code}
> Exception from the output file:
> {code:java}
> 2018-02-05 18:00:48,205 ERROR [PEWorker-1]
> procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception:
> pid=19, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS;
> SplitTableRegionProcedure table=testSplitWithoutPONR,
> parent=57114194fb486a3988b232bcf10eb177,
> daughterA=749aa83c03b8f7c6b642cd73c5b51e43,
> daughterB=a53ec69e8dd2cfa6c0be2b9a7eb271bb
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.splitStoreFiles(SplitTableRegionProcedure.java:617)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createDaughterRegions(SplitTableRegionProcedure.java:541)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:241)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734){code}
> Value of 'htd' is null as it is initialized in the constructor but when the
> object is deserialized its null.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353564#comment-16353564 ] Duo Zhang commented on HBASE-19929: --- It is introduced by HBASE-7329, and now I think it is useless. We removed the WAL marker for flush in HBASE-7329, but now the flush marker is back... Since we have set closed flag to true, a flush will fail when it wants to write the flush marker, no matter whether we are waiting before actually close the writer as there is a check for closed flag in WAL.append. So I tend to remove the DrainBarrier. In general I do not think this will introduce any big problems such as data inconsistency. Thanks. > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19553) Old replica regions should be cleared from AM memory after primary region split or merge
[ https://issues.apache.org/jira/browse/HBASE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353581#comment-16353581 ] Pankaj Kumar commented on HBASE-19553: -- Tried "TestCatalogJanitorInMemoryStates" locally, it is successful. > Old replica regions should be cleared from AM memory after primary region > split or merge > > > Key: HBASE-19553 > URL: https://issues.apache.org/jira/browse/HBASE-19553 > Project: HBase > Issue Type: Bug > Components: master >Reporter: huaxiang sun >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-19553-branch-1-v2.patch, > HBASE-19553-branch-1-v3.patch, HBASE-19553-branch-1-v4.patch, > HBASE-19553-branch-1-v4.patch, HBASE-19553-branch-1.patch > > > Similar to HBASE-18025, the replica parent's info is not removed from master. > Actually I think it can be removed after replica region is split or merged, I > will check the logic and apply one patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19929: -- Assignee: Duo Zhang Fix Version/s: 2.0.0-beta-2 Status: Patch Available (was: Open) > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19929.patch > > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19929: -- Attachment: HBASE-19929.patch > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19929.patch > > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353591#comment-16353591 ] Duo Zhang commented on HBASE-19929: --- Review board link: https://reviews.apache.org/r/65528/ > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19929.patch > > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[
https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353605#comment-16353605
]
Hadoop QA commented on HBASE-19923:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
13s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
14s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
18s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 28s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncTableBatch |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19923 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909382/HBASE-19923.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 94b5d4fc7aa4 4
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353647#comment-16353647
]
Guanghao Zhang commented on HBASE-19942:
The unit test hope start a rpc executor with 0 handler. But in
RpcExecutor.java, the numCallQueues is at least 1, so the handler count is at
least 1, too. And the default call queue type is fifo. So it will start a
FastPathBalancedQueueRpcExecutor. If the handler was already start, then the
callTask will be dispatch to the handler directly and no need add to queue.
{code}
SimpleRpcScheduler scheduler = new SimpleRpcScheduler(schedConf, 0, 0, 0,
priority,
HConstants.QOS_THRESHOLD);
{code}
this.handlerCount = Math.max(handlerCount, this.numCallQueues);
protected int computeNumCallQueues(final int handlerCount, final float
callQueuesHandlersFactor) {
return Math.max(1, Math.round(handlerCount * callQueuesHandlersFactor));
}
{code}
{code}
public boolean dispatch(CallRunner callTask) throws InterruptedException {
FastPathHandler handler = popReadyHandler();
return handler != null? handler.loadCallRunner(callTask):
super.dispatch(callTask);
}
{code}
> Fix flaky TestSimpleRpcScheduler
>
>
> Key: HBASE-19942
> URL: https://issues.apache.org/jira/browse/HBASE-19942
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html]
>
> https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/
>
> h3. Stacktrace
> java.lang.AssertionError at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19942: --- Attachment: HBASE-19942.master.001.patch > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-19942: -- Assignee: Guanghao Zhang > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19942: --- Status: Patch Available (was: Open) > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19923) Reset peer state and config when refresh replication source failed
[ https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19923: --- Attachment: HBASE-19923.master.001.patch > Reset peer state and config when refresh replication source failed > -- > > Key: HBASE-19923 > URL: https://issues.apache.org/jira/browse/HBASE-19923 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19923.master.001.patch, > HBASE-19923.master.001.patch > > > Now we use procedure for replication. When peer state changed, the RS will > read peer state from storage to cache. If RS found the peer state changed, > then it will refresh replication source. If refresh failed, the Master will > retry the procedure. Then RS will read peer state again, but now the peer > state in cache is right. So it don't refresh replication source.. So we > need reset the peer state to old peer state when refresh failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
Guanghao Zhang created HBASE-19944: -- Summary: Fix timeout TestVisibilityLabelsWithCustomVisLabService Key: HBASE-19944 URL: https://issues.apache.org/jira/browse/HBASE-19944 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] h3. Error Message test timed out after 6 milliseconds h3. Stacktrace org.junit.runners.model.TestTimedOutException: test timed out after 6 milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19944: --- Assignee: Guanghao Zhang Status: Patch Available (was: Open) > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19944: --- Attachment: HBASE-19944.master.001.patch > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353684#comment-16353684 ] Peter Somogyi commented on HBASE-19944: --- +1 > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[
https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353697#comment-16353697
]
Hadoop QA commented on HBASE-19935:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.wal.TestSyncReplicationWALProvider |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19935 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909387/HBASE-19935.HBASE-19064.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 48817f4cc212 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 7d5e6f7d05 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11410/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11410/testReport/ |
| Max. process+thread count | 5308 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11410/console |
| Powered
[jira] [Updated] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19935: --- Attachment: HBASE-19935.HBASE-19064.002.patch > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch, > HBASE-19935.HBASE-19064.002.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19937) Enable rsgroup NPE in CreateTableProcedure
[ https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353728#comment-16353728 ] Guanghao Zhang commented on HBASE-19937: Why create other system table don't throw NPE? And they will use RSGroupBasedLoadBalancer, too. > Enable rsgroup NPE in CreateTableProcedure > -- > > Key: HBASE-19937 > URL: https://issues.apache.org/jira/browse/HBASE-19937 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0-beta-2 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: HBASE-19937.branch-2.001.patch > > > When enable rsgroup, it may throws NPE as follows, > 2018-02-02,16:12:45,688 ERROR > org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught > runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS; > CreateTableProcedure table=hbase:rsgroup > java.lang.NullPointerException > at > org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254) > at > org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162) > at > org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603) > at > org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108) > at > org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51) > at > org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182) > at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738) > > As a result of CreateTableProcedure.rollbackState, it may then print logs > warning TableExistsException as follows, > 2018-02-02,16:12:55,503 WARN > org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: > Failed to perform check > java.io.IOException: Failed to create group table. > org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup > at > org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877) > > After some auto-retries, it loops running the thread RSGroupStartupWorker, > will print logs as follows, > 2018-02-02,16:23:17,626 INFO > org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: > RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, > rootMetaFound=true > 2018-02-02,16:23:17,730 INFO > org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: > RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, > rootMetaFound=true > 2018-02-02,16:23:17,834 INFO > org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: > RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, > rootMetaFound=true > 2018-02-02,16:23:17,937 INFO > org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker: > RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0, > rootMetaFound=true > > And using shells of rsgroup, it will tips that currently is in "offline mode". > > The reason of this problem is that CreateTableProcedure used > RSGroupBasedLoadBalancer, who has member variables initialized depending on > return of CreateTableProcedure. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19935: --- Attachment: HBASE-19935.HBASE-19064.003.patch > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch, > HBASE-19935.HBASE-19064.002.patch, HBASE-19935.HBASE-19064.003.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353737#comment-16353737 ] Guanghao Zhang commented on HBASE-19935: Not allow config cfs for replication in 003 patch. If a table is replicated, then all cfs should be replicated. > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch, > HBASE-19935.HBASE-19064.002.patch, HBASE-19935.HBASE-19064.003.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[
https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353739#comment-16353739
]
Hadoop QA commented on HBASE-19929:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} hbase-common: The patch generated 0 new + 0
unchanged - 5 fixed = 0 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} hbase-server: The patch generated 0 new + 132
unchanged - 2 fixed = 132 total (was 134) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
18s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 48s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}145m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestShutdownWhileWALBroken |
| | hadoop.hbase.regionserver.TestWALLockup |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19929 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909394/HBASE-19929.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e2d9b20c1e0f 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven
[jira] [Commented] (HBASE-19941) Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests
[ https://issues.apache.org/jira/browse/HBASE-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353765#comment-16353765 ] Hudson commented on HBASE-19941: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4536 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4536/]) HBASE-19941 Flaky TestCreateTableProcedure times out in nightly, needs (stack: rev 3fa27c2382a70f692eb36d191228f9e796800ffb) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java > Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests > > > Key: HBASE-19941 > URL: https://issues.apache.org/jira/browse/HBASE-19941 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19941.branch-2.001.patch > > > Currently its categorized as MediumTests but sometimes running all test in > this class take more than 180 seconds. Here is the comparison of runtimes > between local runs (on my dev machine) and in nightly runs: > ||Test||Local (seconds)||Nihgtly (seconds)|| > |testSimpleCreateWithSplits|~1.5|~12| > |testRollbackAndDoubleExecutionOnMobTable|~4.7|~21| > |testSimpleCreate|~1.7|~11| > |testRollbackAndDoubleExecution|~4.3|~18| > |testMRegions|~26.4|Timed out after 90 seconds| > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353770#comment-16353770 ] Duo Zhang commented on HBASE-19929: --- Let me check... > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19929.patch > > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353769#comment-16353769 ] Duo Zhang commented on HBASE-19935: --- I think the isSyncReplication method could be added to ReplicationPeerConfig directly? For other parts, +1. > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch, > HBASE-19935.HBASE-19064.002.patch, HBASE-19935.HBASE-19064.003.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[
https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353778#comment-16353778
]
Hadoop QA commented on HBASE-19923:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 10s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m
15s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19923 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909403/HBASE-19923.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c2b62fb2c84e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3fa27c2382 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Def
[jira] [Commented] (HBASE-19934) HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting
[
https://issues.apache.org/jira/browse/HBASE-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353782#comment-16353782
]
Ted Yu commented on HBASE-19934:
Toshihiro:
Can you use dev-support/submit-patch.py to generate patch for branch-1 ?
> HBaseSnapshotException when read replicas is enabled and online snapshot is
> taken after region splitting
>
>
> Key: HBASE-19934
> URL: https://issues.apache.org/jira/browse/HBASE-19934
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19934-UT.patch, HBASE-19934-branch-1.patch,
> HBASE-19934-v2.patch, HBASE-19934-v3.patch, HBASE-19934-v3.patch,
> HBASE-19934.patch, HBASE-19934.patch, HBASE-19934.patch, HBASE-19934.patch
>
>
> Investigating HBASE-19893, I'm encountering another issue.
> Steps to reproduce are as follows:
> 1. Create a table
> {code:java}
> create "test", "cf", {REGION_REPLICATION => 2}{code}
> 2. Load data to the table
> {code:java}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}{code}
> 3. Split the table
> {code:java}
> split "test"{code}
> 4. Take a snapshot for the table
> {code:java}
> snapshot "test", "snap"{code}
> And I encountered the following error:
> {code:java}
> hbase(main):004:0> snapshot "test", "snap"
> ERROR: org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
> ss=snap table=test type=FLUSH } had an error. Procedure snap { waiting=[]
> done=[] }
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:379)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1144)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
> Failed taking snapshot { ss=snap table=test type=FLUSH } due to
> exception:Manifest region info {ENCODED => b910488a686644a7c1c85246d0d123d5,
> NAME => 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.',
> STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID =>
> 1}doesn't match expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a,
> NAME => 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY =>
> '', ENDKEY => '', OFFLINE => true, SPLIT =>
> true}:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Manifest
> region info {ENCODED => b910488a686644a7c1c85246d0d123d5, NAME =>
> 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 1}doesn't match
> expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a, NAME =>
> 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true}
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:82)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:306)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:368)
> ... 6 more
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
> Manifest region info {ENCODED => b910488a686644a7c1c85246d0d123d5, NAME =>
> 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 1}doesn't match
> expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a, NAME =>
> 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true}
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegionInfo(MasterSnapshotVerifier.java:223)
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegions(MasterSnapshotVerifier.java:201)
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:119)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:202)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353788#comment-16353788 ] Ted Yu commented on HBASE-19942: lgtm Please format the above comment properly. > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19929: -- Attachment: HBASE-19929-v1.patch > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19929-v1.patch, HBASE-19929.patch > > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[ https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353793#comment-16353793 ] Duo Zhang commented on HBASE-19929: --- Rmoved TestWALLockup.testLockup16960, as now the LogRoller will close WAL first before abort RS so we can not append any more... Make the new test also test FSHLog. > Call RS.stop on a session expired RS may hang > - > > Key: HBASE-19929 > URL: https://issues.apache.org/jira/browse/HBASE-19929 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19929-v1.patch, HBASE-19929.patch > > > See the discussion in HBASE-19927. The problem is that, for a normal stop we > will try to close all the regions and wait until they are all closed. But if > the RS has already session expired, master will start the failover work which > will move the WAL directory, and then we will be stuck in writing flush > marker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[
https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353795#comment-16353795
]
Hadoop QA commented on HBASE-19944:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
4s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
0s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
5s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}120m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19944 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909407/HBASE-19944.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0a702ac4c32e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3fa27c2382 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11414/testReport/ |
| Max. process+thread count | 5337 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HB
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[ https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353798#comment-16353798 ] Duo Zhang commented on HBASE-19923: --- I think it will be more stable if we use a success flag and reset it in a finally block? > Reset peer state and config when refresh replication source failed > -- > > Key: HBASE-19923 > URL: https://issues.apache.org/jira/browse/HBASE-19923 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19923.master.001.patch, > HBASE-19923.master.001.patch > > > Now we use procedure for replication. When peer state changed, the RS will > read peer state from storage to cache. If RS found the peer state changed, > then it will refresh replication source. If refresh failed, the Master will > retry the procedure. Then RS will read peer state again, but now the peer > state in cache is right. So it don't refresh replication source.. So we > need reset the peer state to old peer state when refresh failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353804#comment-16353804 ] Duo Zhang commented on HBASE-19944: --- +1. The compile error is in hbase-thrift which is not related. > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353808#comment-16353808 ] Duo Zhang commented on HBASE-19944: --- Test locally, there is no compile error. Let me help committing. > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19944: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0-beta-2 Status: Resolved (was: Patch Available) Pushed to master and branch-2. > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353822#comment-16353822
]
Hadoop QA commented on HBASE-19942:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}130m 49s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.access.TestZKPermissionWatcher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19942 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909402/HBASE-19942.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3a2a3df0f83e 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3fa27c2382 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11412/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11412/testReport/ |
| Max. process+thread count | 5263 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11412/console |
| Powered by | Apache Yetus 0.7.0 http:
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[
https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353867#comment-16353867
]
Hadoop QA commented on HBASE-19935:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
44s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 14s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19935 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909417/HBASE-19935.HBASE-19064.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2ae8a3ef5fc3 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 7d5e6f7d05 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11415/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11415/testReport/ |
| Max. process+thread count | 4901 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11415/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Comment Edited] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353647#comment-16353647
]
Guanghao Zhang edited comment on HBASE-19942 at 2/6/18 1:31 PM:
The unit test hope start a rpc executor with 0 handler. But in
RpcExecutor.java, the numCallQueues is at least 1, so the handler count is at
least 1, too. And the default call queue type is fifo. So it will start a
FastPathBalancedQueueRpcExecutor. If the handler was already start, then the
callTask will be dispatch to the handler directly and no need add to queue.
{code:java}
SimpleRpcScheduler scheduler = new SimpleRpcScheduler(schedConf, 0, 0, 0,
priority,
HConstants.QOS_THRESHOLD);
{code}
{code:java}
this.handlerCount = Math.max(handlerCount, this.numCallQueues);
protected int computeNumCallQueues(final int handlerCount, final float
callQueuesHandlersFactor)
{ return Math.max(1, Math.round(handlerCount * callQueuesHandlersFactor));
}{code}
{code:java}
public boolean dispatch(CallRunner callTask) throws InterruptedException
{ FastPathHandler handler = popReadyHandler(); return handler != null?
handler.loadCallRunner(callTask): super.dispatch(callTask); }{code}
was (Author: zghaobac):
The unit test hope start a rpc executor with 0 handler. But in
RpcExecutor.java, the numCallQueues is at least 1, so the handler count is at
least 1, too. And the default call queue type is fifo. So it will start a
FastPathBalancedQueueRpcExecutor. If the handler was already start, then the
callTask will be dispatch to the handler directly and no need add to queue.
{code}
SimpleRpcScheduler scheduler = new SimpleRpcScheduler(schedConf, 0, 0, 0,
priority,
HConstants.QOS_THRESHOLD);
{code}
this.handlerCount = Math.max(handlerCount, this.numCallQueues);
protected int computeNumCallQueues(final int handlerCount, final float
callQueuesHandlersFactor) {
return Math.max(1, Math.round(handlerCount * callQueuesHandlersFactor));
}
{code}
{code}
public boolean dispatch(CallRunner callTask) throws InterruptedException {
FastPathHandler handler = popReadyHandler();
return handler != null? handler.loadCallRunner(callTask):
super.dispatch(callTask);
}
{code}
> Fix flaky TestSimpleRpcScheduler
>
>
> Key: HBASE-19942
> URL: https://issues.apache.org/jira/browse/HBASE-19942
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-19942.master.001.patch
>
>
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html]
>
> https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/
>
> h3. Stacktrace
> java.lang.AssertionError at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353647#comment-16353647
]
Guanghao Zhang edited comment on HBASE-19942 at 2/6/18 1:32 PM:
The unit test hope start a rpc executor with 0 handler. But in
RpcExecutor.java, the numCallQueues is at least 1, so the handler count is at
least 1, too. And the default call queue type is fifo. So it will start a
FastPathBalancedQueueRpcExecutor. If the handler was already start, then the
callTask will be dispatch to the handler directly and no need add to queue.
{code:java}
SimpleRpcScheduler scheduler = new SimpleRpcScheduler(schedConf, 0, 0, 0,
priority,
HConstants.QOS_THRESHOLD);
{code}
{code:java}
this.handlerCount = Math.max(handlerCount, this.numCallQueues);
protected int computeNumCallQueues(final int handlerCount, final float
callQueuesHandlersFactor) {
return Math.max(1, Math.round(handlerCount * callQueuesHandlersFactor));
}{code}
{code:java}
public boolean dispatch(CallRunner callTask) throws InterruptedException {
FastPathHandler handler = popReadyHandler(); return handler != null?
handler.loadCallRunner(callTask): super.dispatch(callTask);
}{code}
was (Author: zghaobac):
The unit test hope start a rpc executor with 0 handler. But in
RpcExecutor.java, the numCallQueues is at least 1, so the handler count is at
least 1, too. And the default call queue type is fifo. So it will start a
FastPathBalancedQueueRpcExecutor. If the handler was already start, then the
callTask will be dispatch to the handler directly and no need add to queue.
{code:java}
SimpleRpcScheduler scheduler = new SimpleRpcScheduler(schedConf, 0, 0, 0,
priority,
HConstants.QOS_THRESHOLD);
{code}
{code:java}
this.handlerCount = Math.max(handlerCount, this.numCallQueues);
protected int computeNumCallQueues(final int handlerCount, final float
callQueuesHandlersFactor)
{ return Math.max(1, Math.round(handlerCount * callQueuesHandlersFactor));
}{code}
{code:java}
public boolean dispatch(CallRunner callTask) throws InterruptedException
{ FastPathHandler handler = popReadyHandler(); return handler != null?
handler.loadCallRunner(callTask): super.dispatch(callTask); }{code}
> Fix flaky TestSimpleRpcScheduler
>
>
> Key: HBASE-19942
> URL: https://issues.apache.org/jira/browse/HBASE-19942
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-19942.master.001.patch
>
>
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html]
>
> https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/
>
> h3. Stacktrace
> java.lang.AssertionError at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19942: --- Attachment: HBASE-19942.branch-2.001.patch > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.branch-2.001.patch, > HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353882#comment-16353882 ] Guanghao Zhang commented on HBASE-19942: Add patch to trigger Hadoop QA for branch-2. > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.branch-2.001.patch, > HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[
https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353887#comment-16353887
]
Guanghao Zhang commented on HBASE-19935:
{quote}I think the isSyncReplication method could be added to
ReplicationPeerConfig directly?
{quote}
Sounds good. Will attach a new patch.
> Only allow table replication for sync replication for now
> -
>
> Key: HBASE-19935
> URL: https://issues.apache.org/jira/browse/HBASE-19935
> Project: HBase
> Issue Type: Sub-task
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-19935.HBASE-19064.001.patch,
> HBASE-19935.HBASE-19064.002.patch, HBASE-19935.HBASE-19064.003.patch
>
>
> Add pre check to only allow table replication for now, no namespace, or
> replicate all and exclusion.
> This is used to reduce the difficulty for implementing the sync replication
> state transition as we need to reopen all the related regions.
> We can add the support for these features later.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19528) Major Compaction Tool
[ https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353888#comment-16353888 ] stack commented on HBASE-19528: --- I pushed it to branch-1 [~churromorales] Sorry it was such a pain landing it. Need to figure why the transitory compile failure against older hadoops. Thanks for persevering. > Major Compaction Tool > -- > > Key: HBASE-19528 > URL: https://issues.apache.org/jira/browse/HBASE-19528 > Project: HBase > Issue Type: New Feature >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, > HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, > HBASE-19528.v1.patch, HBASE-19528.v2.branch-1 (1).patch, > HBASE-19528.v2.branch-1 (1).patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch > > > The basic overview of how this tool works is: > Parameters: > Table > Stores > ClusterConcurrency > Timestamp > So you input a table, desired concurrency and the list of stores you wish to > major compact. The tool first checks the filesystem to see which stores need > compaction based on the timestamp you provide (default is current time). It > takes that list of stores that require compaction and executes those requests > concurrently with at most N distinct RegionServers compacting at a given > time. Each thread waits for the compaction to complete before moving to the > next queue. If a region split, merge or move happens this tool ensures those > regions get major compacted as well. > This helps us in two ways, we can limit how much I/O bandwidth we are using > for major compaction cluster wide and we are guaranteed after the tool > completes that all requested compactions complete regardless of moves, merges > and splits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19528) Major Compaction Tool
[ https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19528: -- Fix Version/s: 1.5.0 > Major Compaction Tool > -- > > Key: HBASE-19528 > URL: https://issues.apache.org/jira/browse/HBASE-19528 > Project: HBase > Issue Type: New Feature >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 1.5.0, 2.0.0-beta-2 > > Attachments: 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, > HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, > HBASE-19528.v1.patch, HBASE-19528.v2.branch-1 (1).patch, > HBASE-19528.v2.branch-1 (1).patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch > > > The basic overview of how this tool works is: > Parameters: > Table > Stores > ClusterConcurrency > Timestamp > So you input a table, desired concurrency and the list of stores you wish to > major compact. The tool first checks the filesystem to see which stores need > compaction based on the timestamp you provide (default is current time). It > takes that list of stores that require compaction and executes those requests > concurrently with at most N distinct RegionServers compacting at a given > time. Each thread waits for the compaction to complete before moving to the > next queue. If a region split, merge or move happens this tool ensures those > regions get major compacted as well. > This helps us in two ways, we can limit how much I/O bandwidth we are using > for major compaction cluster wide and we are guaranteed after the tool > completes that all requested compactions complete regardless of moves, merges > and splits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19528) Major Compaction Tool
[ https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19528: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Resolving. > Major Compaction Tool > -- > > Key: HBASE-19528 > URL: https://issues.apache.org/jira/browse/HBASE-19528 > Project: HBase > Issue Type: New Feature >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 1.5.0, 2.0.0-beta-2 > > Attachments: 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, > HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, > HBASE-19528.v1.patch, HBASE-19528.v2.branch-1 (1).patch, > HBASE-19528.v2.branch-1 (1).patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch > > > The basic overview of how this tool works is: > Parameters: > Table > Stores > ClusterConcurrency > Timestamp > So you input a table, desired concurrency and the list of stores you wish to > major compact. The tool first checks the filesystem to see which stores need > compaction based on the timestamp you provide (default is current time). It > takes that list of stores that require compaction and executes those requests > concurrently with at most N distinct RegionServers compacting at a given > time. Each thread waits for the compaction to complete before moving to the > next queue. If a region split, merge or move happens this tool ensures those > regions get major compacted as well. > This helps us in two ways, we can limit how much I/O bandwidth we are using > for major compaction cluster wide and we are guaranteed after the tool > completes that all requested compactions complete regardless of moves, merges > and splits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19935) Only allow table replication for sync replication for now
[ https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19935: --- Attachment: HBASE-19935.HBASE-19064.004.patch > Only allow table replication for sync replication for now > - > > Key: HBASE-19935 > URL: https://issues.apache.org/jira/browse/HBASE-19935 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19935.HBASE-19064.001.patch, > HBASE-19935.HBASE-19064.002.patch, HBASE-19935.HBASE-19064.003.patch, > HBASE-19935.HBASE-19064.004.patch > > > Add pre check to only allow table replication for now, no namespace, or > replicate all and exclusion. > This is used to reduce the difficulty for implementing the sync replication > state transition as we need to reopen all the related regions. > We can add the support for these features later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[
https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353894#comment-16353894
]
Hadoop QA commented on HBASE-19935:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
43s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
17s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 45s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19935 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909420/HBASE-19935.HBASE-19064.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7e5e1b1f1759 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 7d5e6f7d05 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11416/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11416/testReport/ |
| Max. process+thread count | 4771 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11416/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> O
[jira] [Updated] (HBASE-19930) fix ImmutableMemStoreLAB#forceCopyOfBigCellInto
[ https://issues.apache.org/jira/browse/HBASE-19930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gali Sheffi updated HBASE-19930: Attachment: HBASE-19930-V03.patch > fix ImmutableMemStoreLAB#forceCopyOfBigCellInto > --- > > Key: HBASE-19930 > URL: https://issues.apache.org/jira/browse/HBASE-19930 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Gali Sheffi >Assignee: Gali Sheffi >Priority: Major > Attachments: HBASE-19930-V01.patch, HBASE-19930-V02.patch, > HBASE-19930-V03.patch > > > This issue is about fixing ImmutableMemStoreLAB#forceCopyOfBigCellInto. > Following a comment in HBASE-19133 regarding a bug in > ImmutableMemStoreLAB#forceCopyOfBigCellInto (assuming this method is never > called for an ImmutableMemStoreLAB, and just throwing an > IllegalStateException whenever called), the forceCopyOfBigCellInto method now > performs the copy of big cells on the first MSLABImpl in its mslabs > linked-list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353916#comment-16353916 ] stack commented on HBASE-19942: --- +1 on patch. That must have been tough to figure out. Thanks [~zghaobac] > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19942.branch-2.001.patch, > HBASE-19942.master.001.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19930) fix ImmutableMemStoreLAB#forceCopyOfBigCellInto
[ https://issues.apache.org/jira/browse/HBASE-19930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353920#comment-16353920 ] Gali Sheffi commented on HBASE-19930: - Hi all, I added a link to the Review Board. You are more than welcome to go over the test and share your comments. Thanks! > fix ImmutableMemStoreLAB#forceCopyOfBigCellInto > --- > > Key: HBASE-19930 > URL: https://issues.apache.org/jira/browse/HBASE-19930 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Gali Sheffi >Assignee: Gali Sheffi >Priority: Major > Attachments: HBASE-19930-V01.patch, HBASE-19930-V02.patch, > HBASE-19930-V03.patch > > > This issue is about fixing ImmutableMemStoreLAB#forceCopyOfBigCellInto. > Following a comment in HBASE-19133 regarding a bug in > ImmutableMemStoreLAB#forceCopyOfBigCellInto (assuming this method is never > called for an ImmutableMemStoreLAB, and just throwing an > IllegalStateException whenever called), the forceCopyOfBigCellInto method now > performs the copy of big cells on the first MSLABImpl in its mslabs > linked-list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19929) Call RS.stop on a session expired RS may hang
[
https://issues.apache.org/jira/browse/HBASE-19929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353929#comment-16353929
]
Hadoop QA commented on HBASE-19929:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
35s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} hbase-common: The patch generated 0 new + 0
unchanged - 5 fixed = 0 total (was 5) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} hbase-server: The patch generated 1 new + 134
unchanged - 8 fixed = 135 total (was 142) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
10s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
8s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
14s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
25s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m
36s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19929 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909429/HBASE-19929-v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle comp
[jira] [Updated] (HBASE-19940) TestMetaShutdownHandler flakey
[ https://issues.apache.org/jira/browse/HBASE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19940: -- Fix Version/s: 2.0.0-beta-2 > TestMetaShutdownHandler flakey > -- > > Key: HBASE-19940 > URL: https://issues.apache.org/jira/browse/HBASE-19940 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19940.master.001.patch > > > Fails 13% of the time. One of the RS won't go down. It has an errant thread > running. Not sure what. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19940) TestMetaShutdownHandler flakey
[
https://issues.apache.org/jira/browse/HBASE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353943#comment-16353943
]
stack commented on HBASE-19940:
---
Add more debug w/ the below:
commit 590bee78dc9cb08a7b2d0eb5205d9dae65a8800a
Author: Michael Stack
Date: Tue Feb 6 06:34:49 2018 -0800
HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: more debug
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
index 172c170b92..0d410fc916 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
@@ -297,10 +297,10 @@ public class JVMClusterUtil {
for (RegionServerThread t : regionservers) {
if (t.isAlive()) {
LOG.warn("RegionServerThreads taking too long to stop,
interrupting; thread dump " +
- "if > three attempts");
+ "if > three attempts: i=" + i);
if (i > 3) {
try {
-Threads.threadDumpingIsAlive(t.getRegionServer().getThread());
+Threads.threadDumpingIsAlive(t);
} catch (InterruptedException e) {
e.printStackTrace();
}
I seem to have been passing the wrong thread we are not getting the wanted
thread dumps out of the hanging RS. Pushed the above on master and branch-2.
> TestMetaShutdownHandler flakey
> --
>
> Key: HBASE-19940
> URL: https://issues.apache.org/jira/browse/HBASE-19940
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19940.master.001.patch
>
>
> Fails 13% of the time. One of the RS won't go down. It has an errant thread
> running. Not sure what.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17449) Add explicit document on different timeout settings
[ https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353961#comment-16353961 ] Peter Somogyi commented on HBASE-17449: --- I started to look into how can we test the different timeouts. My idea is to start up a minicluster, execute different types of requests (mutations, scans, admin operations, etc.) while we add some delay on the server side. What I have already noticed is that even to start up a cluster "hbase.client.operation.timeout" is in use. When I put it to 1ms (I know it is not something you would use in real life scenario) the cluster did not come up. It caused endless retries even though I saw DoNotRetryIOException in the log. What do you think about this approach? > Add explicit document on different timeout settings > --- > > Key: HBASE-17449 > URL: https://issues.apache.org/jira/browse/HBASE-17449 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Yu Li >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0-beta-2 > > > Currently we have more than one timeout settings, mainly includes: > * hbase.rpc.timeout > * hbase.client.operation.timeout > * hbase.client.scanner.timeout.period > And in latest branch-1 or master branch code, we will have two other > properties: > * hbase.rpc.read.timeout > * hbase.rpc.write.timeout > However, in current refguid we don't have explicit instruction on the > difference of these timeout settings (there're explanations for each > property, but no instruction on when to use which) > In my understanding, for RPC layer timeout, or say each rpc call: > * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period > * Other operations: >1. For released versions: controlled by hbase.rpc.timeout >2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, > write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout > if the previous two are not set. > And hbase.client.operation.timeout is a higher-level control counting retry > in, or say the overall control for one user call. > After this JIRA, I hope when users ask questions like "What settings I should > use if I don't want to wait for more than 1 second for a single > put/get/scan.next call", we could give a neat answer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19824) SingleColumnValueFilter returns wrong result when used in shell command
[
https://issues.apache.org/jira/browse/HBASE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353963#comment-16353963
]
Reid Chan commented on HBASE-19824:
---
Feel sorry for late response..
bq. perhaps we can introduce an new filter to address it.
Agree, it will not add complexity to the {{SingleColumnValueFilter}}, which may
break its original behavior and add confusion to users.
And, i think some of users may, as Ted and i did, think this filter does no
filtering at the beginning, indicating that the function they want or expect is
to fetch exactly column value match cells.
I can add that and close this one if no objections.
> SingleColumnValueFilter returns wrong result when used in shell command
> ---
>
> Key: HBASE-19824
> URL: https://issues.apache.org/jira/browse/HBASE-19824
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Ted Yu
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-19824.master.001.patch,
> HBASE-19824.master.002.patch
>
>
> There are two cells in table t1:
> {code}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
> When SingleColumnValueFilter is used in shell command, no filtering was done:
> {code}
> hbase(main):022:0> scan 't1', {FILTER => "SingleColumnValueFilter('f1', 'a1',
> =, 'binary:a2')"}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353979#comment-16353979
]
Edward Bortnikov commented on HBASE-18294:
--
[~eshcar], [~anoop.hbase], [~ram_krish], maybe we can take the following
approach to your dispute on RB.
The code makes the following config variable definitions:
{code:java}
public static final String HREGION_MEMSTORE_FLUSH_SIZE =
"hbase.hregion.memstore.flush.size";
public static final String HREGION_MEMSTORE_OFFHEAP_FLUSH_SIZE =
"hbase.hregion.memstore.offheap.flush.size";{code}
The former is the legacy flush size threshold, whereas the latter is new.
However, the further treatment is different - HREGION_MEMORY_FLUSH_SIZE is
actually treated as *on-heap* threshold. This is confusing I guess - especially
for admins.
Having said that, we do need separate accounting for on-heap and off-heap
memory, as [~eshcar] explained above. Let me suggest a change that is more
digestible for users imo. Let HREGION_MEMORY_FLUSH_SIZE retain its legacy
meaning (and the 128M default) - namely, the overall max memory the system is
willing to allocate for a store. Furthermore, let's define a new variable,
HREGION_MEMSTORE_OFFHEAP_SIZE_RATIO, to define a fraction of the former that
can be allocated offheap (0, by default).
How about that?
> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch,
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19900) Region-level exception destroy the result of batch
[ https://issues.apache.org/jira/browse/HBASE-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-19900: --- Attachment: HBASE-19900.v2.patch > Region-level exception destroy the result of batch > -- > > Key: HBASE-19900 > URL: https://issues.apache.org/jira/browse/HBASE-19900 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2 > > Attachments: HBASE-19900.v0.patch, HBASE-19900.v1.patch, > HBASE-19900.v2.patch, HBASE-19900.v2.patch > > > 1) decrease action count repeatedly > If the AsyncRequestFuture#waitUntilDone return prematurely, user will get the > incorrect results. Or user will be block by AsyncRequestFuture#waitUntilDone > as the count is never equal with 0. > 2) the successive result will be overwrited > 3) The failed op is added to RetriesExhaustedWithDetailsException repeatedly > AsyncRequestFutureImpl#receiveMultiAction process the action-lever error > first, and then add the region-level exception to each action. Hence, user > may get the various exceptions for the same action (row op) from the > RetriesExhaustedWithDetailsException. > In fact, if both of action-level exception and region-lever exception exist, > they always have the same context. I'm not sure whether that is what > RetriesExhaustedWithDetailsException want. As i see it, we shouldn't have the > duplicate ops in RetriesExhaustedWithDetailsException since that may confuse > users if they catch the RetriesExhaustedWithDetailsException to check the > invalid operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17449) Add explicit document on different timeout settings
[ https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354007#comment-16354007 ] stack commented on HBASE-17449: --- Sounds great. Sounds like interesting issue, retrying though we set operation timeout at 1ms. Any chance of a timeout server-side and then one client-side (or your idea from yesterday of bringing up the cluster in unit test and then making a client but with a Configuration that has the timeouts set down? Thanks [~psomogyi] > Add explicit document on different timeout settings > --- > > Key: HBASE-17449 > URL: https://issues.apache.org/jira/browse/HBASE-17449 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Yu Li >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0-beta-2 > > > Currently we have more than one timeout settings, mainly includes: > * hbase.rpc.timeout > * hbase.client.operation.timeout > * hbase.client.scanner.timeout.period > And in latest branch-1 or master branch code, we will have two other > properties: > * hbase.rpc.read.timeout > * hbase.rpc.write.timeout > However, in current refguid we don't have explicit instruction on the > difference of these timeout settings (there're explanations for each > property, but no instruction on when to use which) > In my understanding, for RPC layer timeout, or say each rpc call: > * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period > * Other operations: >1. For released versions: controlled by hbase.rpc.timeout >2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, > write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout > if the previous two are not set. > And hbase.client.operation.timeout is a higher-level control counting retry > in, or say the overall control for one user call. > After this JIRA, I hope when users ask questions like "What settings I should > use if I don't want to wait for more than 1 second for a single > put/get/scan.next call", we could give a neat answer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19900) Region-level exception destroy the result of batch
[ https://issues.apache.org/jira/browse/HBASE-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354008#comment-16354008 ] Chia-Ping Tsai commented on HBASE-19900: retry. The failed tests pass locally. > Region-level exception destroy the result of batch > -- > > Key: HBASE-19900 > URL: https://issues.apache.org/jira/browse/HBASE-19900 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2 > > Attachments: HBASE-19900.v0.patch, HBASE-19900.v1.patch, > HBASE-19900.v2.patch, HBASE-19900.v2.patch > > > 1) decrease action count repeatedly > If the AsyncRequestFuture#waitUntilDone return prematurely, user will get the > incorrect results. Or user will be block by AsyncRequestFuture#waitUntilDone > as the count is never equal with 0. > 2) the successive result will be overwrited > 3) The failed op is added to RetriesExhaustedWithDetailsException repeatedly > AsyncRequestFutureImpl#receiveMultiAction process the action-lever error > first, and then add the region-level exception to each action. Hence, user > may get the various exceptions for the same action (row op) from the > RetriesExhaustedWithDetailsException. > In fact, if both of action-level exception and region-lever exception exist, > they always have the same context. I'm not sure whether that is what > RetriesExhaustedWithDetailsException want. As i see it, we shouldn't have the > duplicate ops in RetriesExhaustedWithDetailsException since that may confuse > users if they catch the RetriesExhaustedWithDetailsException to check the > invalid operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19506) Support variable sized chunks from ChunkCreator
[ https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gali Sheffi updated HBASE-19506: Attachment: HBASE-19506-V02.patch > Support variable sized chunks from ChunkCreator > --- > > Key: HBASE-19506 > URL: https://issues.apache.org/jira/browse/HBASE-19506 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Priority: Major > Attachments: HBASE-19506-V01.patch, HBASE-19506-V02.patch > > > When CellChunkMap is created it allocates a special index chunk (or chunks) > where array of cell-representations is stored. When the number of > cell-representations is small, it is preferable to allocate a chunk smaller > than a default value which is 2MB. > On the other hand, those "non-standard size" chunks can not be used in pool. > On-demand allocations in off-heap are costly. So this JIRA is about to > investigate the trade of between memory usage and the final performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19824) SingleColumnValueFilter returns wrong result when used in shell command
[
https://issues.apache.org/jira/browse/HBASE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354047#comment-16354047
]
Chia-Ping Tsai commented on HBASE-19824:
{quote}I can add that and close this one if no objections.
{quote}
Ya, I think this issue can be closed as "Won't fix". Let us go to another Jira
to discuss the new filer.
> SingleColumnValueFilter returns wrong result when used in shell command
> ---
>
> Key: HBASE-19824
> URL: https://issues.apache.org/jira/browse/HBASE-19824
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Ted Yu
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-19824.master.001.patch,
> HBASE-19824.master.002.patch
>
>
> There are two cells in table t1:
> {code}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
> When SingleColumnValueFilter is used in shell command, no filtering was done:
> {code}
> hbase(main):022:0> scan 't1', {FILTER => "SingleColumnValueFilter('f1', 'a1',
> =, 'binary:a2')"}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354081#comment-16354081
]
Hadoop QA commented on HBASE-19942:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
7s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 8s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}118m
17s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}153m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19942 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909438/HBASE-19942.branch-2.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b3c86a12f98e 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / fb480775d5 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11418/testReport/ |
| Max. process+thread count | 5034 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11418/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Fix flaky TestSimpleRpcScheduler
>
>
> Key: HBASE-19942
> URL: http
[jira] [Commented] (HBASE-19935) Only allow table replication for sync replication for now
[
https://issues.apache.org/jira/browse/HBASE-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354105#comment-16354105
]
Hadoop QA commented on HBASE-19935:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
5s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 0s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestReplicationDroppedTables |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19935 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909443/HBASE-19935.HBASE-19064.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5d4ff61fe637 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 61c6b8852d |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
[jira] [Commented] (HBASE-19930) fix ImmutableMemStoreLAB#forceCopyOfBigCellInto
[
https://issues.apache.org/jira/browse/HBASE-19930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354107#comment-16354107
]
Hadoop QA commented on HBASE-19930:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19930 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909444/HBASE-19930-V03.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5a6e784ec80a 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 77153d4dc6 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11420/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11420/testReport/ |
| Max. process+thread count | 5096 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11420/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> fix ImmutableMemStoreLAB#forceCopyOfBigCell
[jira] [Updated] (HBASE-19824) SingleColumnValueFilter returns wrong result when used in shell command
[
https://issues.apache.org/jira/browse/HBASE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19824:
---
Resolution: Won't Fix
Assignee: (was: Reid Chan)
Status: Resolved (was: Patch Available)
> SingleColumnValueFilter returns wrong result when used in shell command
> ---
>
> Key: HBASE-19824
> URL: https://issues.apache.org/jira/browse/HBASE-19824
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Ted Yu
>Priority: Major
> Attachments: HBASE-19824.master.001.patch,
> HBASE-19824.master.002.patch
>
>
> There are two cells in table t1:
> {code}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
> When SingleColumnValueFilter is used in shell command, no filtering was done:
> {code}
> hbase(main):022:0> scan 't1', {FILTER => "SingleColumnValueFilter('f1', 'a1',
> =, 'binary:a2')"}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException
[
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-19841:
--
Issue Type: Sub-task (was: Test)
Parent: HBASE-19147
> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt,
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch,
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch,
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch,
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch,
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush and hsync
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being
> picked up by Configuration object. Among the configs from this file, the
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence
> of hflush and hsync. Without this config entry,
> StreamLacksCapabilityException is thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19506) Support variable sized chunks from ChunkCreator
[
https://issues.apache.org/jira/browse/HBASE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354113#comment-16354113
]
Hadoop QA commented on HBASE-19506:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 5 new + 10 unchanged
- 2 fixed = 15 total (was 12) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 42s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestMinVersions |
| | hadoop.hbase.filter.TestFilter |
| | hadoop.hbase.regionserver.TestMemStoreChunkPool |
| | hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan |
| | hadoop.hbase.regionserver.TestStoreFileRefresherChore |
| | hadoop.hbase.filter.TestInvocationRecordFilter |
| | hadoop.hbase.filter.TestFilterFromRegionSide |
| | hadoop.hbase.filter.TestMultipleColumnPrefixFilter |
| | hadoop.hbase.filter.TestDependentColumnFilter |
| | hadoop.hbase.regionserver.TestBlocksScanned |
| | hadoop.hbase.regionserver.TestScanner |
| | hadoop.hbase.filter.TestColumnPrefixFilter |
| | hadoop.hbase.io.hfile.TestScannerSelectionUsingKeyRange |
| | hadoop.hbase.client.TestIntraRowPagination |
| | hadoop.hbase.regionserver.TestScanWithBloomError |
| | hadoop.hbase.regionserver.TestWideScanner |
| | hadoop.hbase.coprocessor.TestCoprocessorInterface |
| | hadoop.hbase.coprocessor.TestRegionObserverStacking |
| | hadoop.hbase.regionserver.TestKeepDeletes |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus
[jira] [Commented] (HBASE-19944) Fix timeout TestVisibilityLabelsWithCustomVisLabService
[ https://issues.apache.org/jira/browse/HBASE-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354122#comment-16354122 ] Hudson commented on HBASE-19944: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4537 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4537/]) HBASE-19944 Fix timeout TestVisibilityLabelsWithCustomVisLabService (zhangduo: rev 77153d4dc69f3cdfc3daeb9b8658c964f3c840d1) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java > Fix timeout TestVisibilityLabelsWithCustomVisLabService > --- > > Key: HBASE-19944 > URL: https://issues.apache.org/jira/browse/HBASE-19944 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19944.master.001.patch > > > [https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1404/testReport/junit/org.apache.hadoop.hbase.security.visibility/TestVisibilityLabelsWithCustomVisLabService/testVisibilityLabelsOnRSRestart/] > > h3. Error Message > test timed out after 6 milliseconds > h3. Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 6 > milliseconds -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException
[
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354151#comment-16354151
]
Mike Drob commented on HBASE-19841:
---
Ok, I understand what you're describing.
Integration tests can run either with a minicluster (which I think will be
fine, since it starts MiniDFS) or with a distributed cluster. I'm thinking that
we need to set the defaultFS back to hdfs in the ITU.createDistributedCluster
call, question is, can we get that info back out?
> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt,
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch,
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch,
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch,
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch,
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush and hsync
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being
> picked up by Configuration object. Among the configs from this file, the
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence
> of hflush and hsync. Without this config entry,
> StreamLacksCapabilityException is thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19528) Major Compaction Tool
[ https://issues.apache.org/jira/browse/HBASE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354157#comment-16354157 ] churro morales commented on HBASE-19528: Thank you for your help [~stack] I really appreciate it > Major Compaction Tool > -- > > Key: HBASE-19528 > URL: https://issues.apache.org/jira/browse/HBASE-19528 > Project: HBase > Issue Type: New Feature >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 1.5.0, 2.0.0-beta-2 > > Attachments: 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Do-nothing-patch.patch, > 0001-HBASE-19528-Major-Compaction-Tool-ADDENDUM.patch, > HBASE-19528.branch-1.patch, HBASE-19528.patch, HBASE-19528.v1.branch-1.patch, > HBASE-19528.v1.patch, HBASE-19528.v2.branch-1 (1).patch, > HBASE-19528.v2.branch-1 (1).patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v2.branch-1.patch, > HBASE-19528.v2.branch-1.patch, HBASE-19528.v8.patch > > > The basic overview of how this tool works is: > Parameters: > Table > Stores > ClusterConcurrency > Timestamp > So you input a table, desired concurrency and the list of stores you wish to > major compact. The tool first checks the filesystem to see which stores need > compaction based on the timestamp you provide (default is current time). It > takes that list of stores that require compaction and executes those requests > concurrently with at most N distinct RegionServers compacting at a given > time. Each thread waits for the compaction to complete before moving to the > next queue. If a region split, merge or move happens this tool ensures those > regions get major compacted as well. > This helps us in two ways, we can limit how much I/O bandwidth we are using > for major compaction cluster wide and we are guaranteed after the tool > completes that all requested compactions complete regardless of moves, merges > and splits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19900) Region-level exception destroy the result of batch
[ https://issues.apache.org/jira/browse/HBASE-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-19900: --- Attachment: HBASE-19900.branch-1.4.v0.patch > Region-level exception destroy the result of batch > -- > > Key: HBASE-19900 > URL: https://issues.apache.org/jira/browse/HBASE-19900 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2 > > Attachments: HBASE-19900.branch-1.4.v0.patch, HBASE-19900.v0.patch, > HBASE-19900.v1.patch, HBASE-19900.v2.patch, HBASE-19900.v2.patch > > > 1) decrease action count repeatedly > If the AsyncRequestFuture#waitUntilDone return prematurely, user will get the > incorrect results. Or user will be block by AsyncRequestFuture#waitUntilDone > as the count is never equal with 0. > 2) the successive result will be overwrited > 3) The failed op is added to RetriesExhaustedWithDetailsException repeatedly > AsyncRequestFutureImpl#receiveMultiAction process the action-lever error > first, and then add the region-level exception to each action. Hence, user > may get the various exceptions for the same action (row op) from the > RetriesExhaustedWithDetailsException. > In fact, if both of action-level exception and region-lever exception exist, > they always have the same context. I'm not sure whether that is what > RetriesExhaustedWithDetailsException want. As i see it, we shouldn't have the > duplicate ops in RetriesExhaustedWithDetailsException since that may confuse > users if they catch the RetriesExhaustedWithDetailsException to check the > invalid operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19945) Separate tests of TestRSGroups into two classes
Ted Yu created HBASE-19945:
--
Summary: Separate tests of TestRSGroups into two classes
Key: HBASE-19945
URL: https://issues.apache.org/jira/browse/HBASE-19945
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
TestRSGroups is annotated as LargeTests already, however, it still times out on
Jenkins:
https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
{code}
org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds
{code}
The above is reproducible on Linux locally.
The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19824) SingleColumnValueFilter returns wrong result when used in shell command
[
https://issues.apache.org/jira/browse/HBASE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354174#comment-16354174
]
Haibo Chen commented on HBASE-19824:
[~yuzhih...@gmail.com] Can you please link the Jira that discusses the new
filter? The behavior described here is likely the same as what I have seen in
ATSv2 with HBase 2.0 (I have not verified with 1.2.6, but it sounds like this
would occur in 1.2.6 as well?).
> SingleColumnValueFilter returns wrong result when used in shell command
> ---
>
> Key: HBASE-19824
> URL: https://issues.apache.org/jira/browse/HBASE-19824
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Ted Yu
>Priority: Major
> Attachments: HBASE-19824.master.001.patch,
> HBASE-19824.master.002.patch
>
>
> There are two cells in table t1:
> {code}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
> When SingleColumnValueFilter is used in shell command, no filtering was done:
> {code}
> hbase(main):022:0> scan 't1', {FILTER => "SingleColumnValueFilter('f1', 'a1',
> =, 'binary:a2')"}
> ROW COLUMN+CELL
> r1 column=f1:a1,
> timestamp=1516313683984, value=a2
> r1 column=f1:b1,
> timestamp=1516313700744, value=b2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354197#comment-16354197
]
stack commented on HBASE-19945:
---
What is it doing that it takes so long to run? Where is it spending the time?
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19900) Region-level exception destroy the result of batch
[
https://issues.apache.org/jira/browse/HBASE-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354200#comment-16354200
]
Hadoop QA commented on HBASE-19900:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} hbase-client: The patch generated 0 new + 30
unchanged - 3 fixed = 30 total (was 33) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
55s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m
47s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m
47s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 19s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.access.TestAccessController3 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19900 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909454/HBASE-19900.v2.patch |
| Optional Tests | asflicense javac javadoc unit fin
[jira] [Updated] (HBASE-19940) TestMetaShutdownHandler flakey
[ https://issues.apache.org/jira/browse/HBASE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19940: -- Attachment: HBASE-19940.master.002.patch > TestMetaShutdownHandler flakey > -- > > Key: HBASE-19940 > URL: https://issues.apache.org/jira/browse/HBASE-19940 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19940.master.001.patch, > HBASE-19940.master.002.patch > > > Fails 13% of the time. One of the RS won't go down. It has an errant thread > running. Not sure what. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19940) TestMetaShutdownHandler flakey
[ https://issues.apache.org/jira/browse/HBASE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354220#comment-16354220 ] stack commented on HBASE-19940: --- Above is still not right. Inside in threadDumpinIsAlive it does a join... but then we do an interrupt on the join'd thread so we exit. I want a thread dump. .002 does explicit thread dump instead of above threadDumpingIsAlive call. Pushed to master and branch-2. > TestMetaShutdownHandler flakey > -- > > Key: HBASE-19940 > URL: https://issues.apache.org/jira/browse/HBASE-19940 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19940.master.001.patch, > HBASE-19940.master.002.patch > > > Fails 13% of the time. One of the RS won't go down. It has an errant thread > running. Not sure what. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354221#comment-16354221
]
Andrew Purtell commented on HBASE-19945:
Agree with Stack. Timeouts I have seen are due to minicluster failing to come
up. Often indicates an initialization/assignment problem. Splitting the test
won't help for that. We can split the tests, but we need to know more.
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19934) HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting
[
https://issues.apache.org/jira/browse/HBASE-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19934:
-
Attachment: HBASE-19934.branch-1.001.patch
> HBaseSnapshotException when read replicas is enabled and online snapshot is
> taken after region splitting
>
>
> Key: HBASE-19934
> URL: https://issues.apache.org/jira/browse/HBASE-19934
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19934-UT.patch, HBASE-19934-branch-1.patch,
> HBASE-19934-v2.patch, HBASE-19934-v3.patch, HBASE-19934-v3.patch,
> HBASE-19934.branch-1.001.patch, HBASE-19934.patch, HBASE-19934.patch,
> HBASE-19934.patch, HBASE-19934.patch
>
>
> Investigating HBASE-19893, I'm encountering another issue.
> Steps to reproduce are as follows:
> 1. Create a table
> {code:java}
> create "test", "cf", {REGION_REPLICATION => 2}{code}
> 2. Load data to the table
> {code:java}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}{code}
> 3. Split the table
> {code:java}
> split "test"{code}
> 4. Take a snapshot for the table
> {code:java}
> snapshot "test", "snap"{code}
> And I encountered the following error:
> {code:java}
> hbase(main):004:0> snapshot "test", "snap"
> ERROR: org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
> ss=snap table=test type=FLUSH } had an error. Procedure snap { waiting=[]
> done=[] }
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:379)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.isSnapshotDone(MasterRpcServices.java:1144)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException via
> Failed taking snapshot { ss=snap table=test type=FLUSH } due to
> exception:Manifest region info {ENCODED => b910488a686644a7c1c85246d0d123d5,
> NAME => 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.',
> STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID =>
> 1}doesn't match expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a,
> NAME => 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY =>
> '', ENDKEY => '', OFFLINE => true, SPLIT =>
> true}:org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Manifest
> region info {ENCODED => b910488a686644a7c1c85246d0d123d5, NAME =>
> 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 1}doesn't match
> expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a, NAME =>
> 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true}
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:82)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:306)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:368)
> ... 6 more
> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException:
> Manifest region info {ENCODED => b910488a686644a7c1c85246d0d123d5, NAME =>
> 'test,,1517808523837_0001.b910488a686644a7c1c85246d0d123d5.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true, REPLICA_ID => 1}doesn't match
> expected region:{ENCODED => ef8665859c0b19927b7dc127ec10120a, NAME =>
> 'test,,1517808523837.ef8665859c0b19927b7dc127ec10120a.', STARTKEY => '',
> ENDKEY => '', OFFLINE => true, SPLIT => true}
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegionInfo(MasterSnapshotVerifier.java:223)
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifyRegions(MasterSnapshotVerifier.java:201)
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:119)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:202)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354262#comment-16354262
]
Ted Yu commented on HBASE-19945:
Here was the rundown for the tests:
setup: 16s
testDefaultNamespaceCreateAndAssign : 7s
testNamespaceCreateAndAssign : 8s
testCloneSnapshot : 10s
testBasicStartUp : < 1s
testMisplacedRegions : 14s
testGroupInfoMultiAccessing : < 1s
testNamespaceConstraint : 2s
testCreateMultiRegion : 7s
testMoveServersAndTables : 20s
testTableMoveTruncateAndDrop : 20s
testValidGroupNames : < 1s
testSimpleRegionServerMove : 5s
testFailRemoveGroup : 10s
testCreateAndDrop : 4.5s
testMultiTableMove : 13s
testBogusArgs : < 1s
testGroupBalance : 17s
testDisabledTableMove : 7s
testRemoveServers : 7s
testMoveServers : 3s
testClearDeadServers : 3s
testKillRS : this was the last one, unfinished > 3s
The total runtime of the above combined was already 180s
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19945:
---
Attachment: 19945.v1.txt
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt
>
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19945:
---
Status: Patch Available (was: Open)
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt
>
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354271#comment-16354271
]
Ted Yu commented on HBASE-19945:
Here is duration for the two test classes (on Mac):
TestRSGroups : 2 minutes
TestRSGroups2 : 149 seconds
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt
>
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354272#comment-16354272
]
stack commented on HBASE-19945:
---
LargeTest is 10minutes. My understanding is that the timeout is per test in a
class, not compound:
http://junit.org/junit4/javadoc/4.12/org/junit/rules/Timeout.html. So, which
method is timing out and what is it doing that takes 10minutes?
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt
>
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19945:
---
Attachment: 19945.v2.txt
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt, 19945.v2.txt
>
>
> TestRSGroups is annotated as LargeTests already, however, it still times out
> on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19945:
---
Description:
TestRSGroups is annotated as MediumTests. It times out on Jenkins:
https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
{code}
org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds
{code}
The above is reproducible on Linux locally.
The tests from TestRSGroups should be separated into two classes.
was:
TestRSGroups is annotated as LargeTests already, however, it still times out on
Jenkins:
https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
{code}
org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds
{code}
The above is reproducible on Linux locally.
The tests from TestRSGroups should be separated into two classes.
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt, 19945.v2.txt
>
>
> TestRSGroups is annotated as MediumTests. It times out on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354304#comment-16354304
]
Ted Yu commented on HBASE-19945:
I corrected description for TestRSGroups in the JIRA.
Patch v2 changes the test to LargeTests.
Depending on which patch is adopted, I can adjust the title accordingly.
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt, 19945.v2.txt
>
>
> TestRSGroups is annotated as MediumTests. It times out on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354313#comment-16354313
]
Ted Yu commented on HBASE-19945:
>From pom.xml :
{code}
{code}
With total runtime of ~ 270 seconds, the single test approach seems not as good
as the two test approach where the two tests run in parallel.
> Separate tests of TestRSGroups into two classes
> ---
>
> Key: HBASE-19945
> URL: https://issues.apache.org/jira/browse/HBASE-19945
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19945.v1.txt, 19945.v2.txt
>
>
> TestRSGroups is annotated as MediumTests. It times out on Jenkins:
> https://builds.apache.org/job/HBase-Trunk_matrix/4537/jdk=JDK%201.8%20(latest),label=(Hadoop%20&&%20!H5)/testReport/junit/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 180
> seconds
> {code}
> The above is reproducible on Linux locally.
> The tests from TestRSGroups should be separated into two classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19946) TestPerColumnFamilyFlush and TestWalAndCompactingMemStoreFlush fail against h3
Mike Drob created HBASE-19946: - Summary: TestPerColumnFamilyFlush and TestWalAndCompactingMemStoreFlush fail against h3 Key: HBASE-19946 URL: https://issues.apache.org/jira/browse/HBASE-19946 Project: HBase Issue Type: Task Components: test Reporter: Mike Drob Assignee: Mike Drob Fix For: 2.0.0-beta-2 These tests started failing against hadoop 3 after HBASE-19841 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19841) Tests against hadoop3 fail with StreamLacksCapabilityException
[
https://issues.apache.org/jira/browse/HBASE-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354315#comment-16354315
]
Mike Drob commented on HBASE-19841:
---
I'd rather not revert this, as I think there is some gymnastics we can do to
make ITU work and I feel like the changes are a huge improvement to HTU. Let me
try some testing locally and get back to you today. Sorry for causing you
troubles..
> Tests against hadoop3 fail with StreamLacksCapabilityException
> --
>
> Key: HBASE-19841
> URL: https://issues.apache.org/jira/browse/HBASE-19841
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19841.007.patch, 19841.06.patch, 19841.v0.txt,
> 19841.v1.txt, HBASE-19841.v10.patch, HBASE-19841.v11.patch,
> HBASE-19841.v11.patch, HBASE-19841.v2.patch, HBASE-19841.v3.patch,
> HBASE-19841.v4.patch, HBASE-19841.v5.patch, HBASE-19841.v7.patch,
> HBASE-19841.v8.patch, HBASE-19841.v8.patch, HBASE-19841.v8.patch,
> HBASE-19841.v9.patch
>
>
> The following can be observed running against hadoop3:
> {code}
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush and hsync
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107)
> at
> org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
> {code}
> This was due to hbase-server/src/test/resources/hbase-site.xml not being
> picked up by Configuration object. Among the configs from this file, the
> value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence
> of hflush and hsync. Without this config entry,
> StreamLacksCapabilityException is thrown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19946) TestPerColumnFamilyFlush and TestWalAndCompactingMemStoreFlush fail against h3
[ https://issues.apache.org/jira/browse/HBASE-19946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-19946: -- Attachment: HBASE-19946.patch > TestPerColumnFamilyFlush and TestWalAndCompactingMemStoreFlush fail against h3 > -- > > Key: HBASE-19946 > URL: https://issues.apache.org/jira/browse/HBASE-19946 > Project: HBase > Issue Type: Task > Components: test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19946.patch > > > These tests started failing against hadoop 3 after HBASE-19841 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19945) Separate tests of TestRSGroups into two classes
[
https://issues.apache.org/jira/browse/HBASE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354324#comment-16354324
]
Hadoop QA commented on HBASE-19945:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} hbase-rsgroup: The patch generated 4 new + 0 unchanged
- 0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 44s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
50s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19945 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909481/19945.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7500615e75f3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 001f9cc5ea |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11425/artifact/patchprocess/diff-checkstyle-hbase-rsgroup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11425/testReport/ |
| M
