[jira] [Commented] (HBASE-19735) Create a minimal "client" tarball installation
[
https://issues.apache.org/jira/browse/HBASE-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328402#comment-16328402
]
Hadoop QA commented on HBASE-19735:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}154m
47s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
43s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
10s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}149m
14s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}347m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19735 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905189/HBASE-19735.001.branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile shellcheck shelldocs |
| uname | Linux 420de0f5d298 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 20afaca769 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| shellcheck | v0.4.4 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/11076/artifact/patchprocess/whitespace-tabs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11076/testReport/ |
| modules | C: hbase-client-assembly . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11076/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
[jira] [Updated] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[ https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19792: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks [~stack] for reviewing. > TestReplicationSmallTests.testDisableEnable fails > - > > Key: HBASE-19792 > URL: https://issues.apache.org/jira/browse/HBASE-19792 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19792.patch, HBASE-19792.patch, HBASE-19792.patch, > org.apache.hadoop.hbase.replication.TestReplicationSmallTests-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328342#comment-16328342
]
Sergey Soldatov commented on HBASE-19805:
-
an update. An additional check for empty region location as well as for CLOSING
state in checkSplittable method helps to avoid multiple splits for the same
region as well as it solves this exception. But now I see that it may to get
stuck somewhere during the split execution, so I'm trying to find the reason
and fix it before submitting a patch.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
>
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328340#comment-16328340
]
Hadoop QA commented on HBASE-19770:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 58m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 98m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
52s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
6s{color} | {color:red} The patch generated 3 new + 48 unchanged - 2 fixed = 51
total (was 50) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
3s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}112m 50s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}281m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.snapshot.TestRegionSnapshotTask |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19770 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905736/HBASE-19770.001.branch-2.patch
|
| Optional Tests | asflicense rubocop ruby_lint javac javadoc unit |
| uname | Linux 720c4193d002 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 20afaca769 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.52.1 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/11077/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11077/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11077/testReport/ |
| modules | C: hbase-shell . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11077/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch, HBASE-19770.003.branch-2.patch
>
>
> Another good find by our Romil.
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328332#comment-16328332 ] stack commented on HBASE-19598: --- Test helped [~balazs.meszaros] .001 Root issue is that Master was stuck in waitForMasterActive, regions were being assigned to Master, and the metrics we were expecting were incorrect (if the killed regionserver was hosting user-space and hbase:meta region). Master never left waitForMasterActive because it never checked state of the clusterUp flag. The test here was aborting regionserver and then just exiting. The minihbasecluster shutdown sets the cluster down flag but we were never looking at it so Master thread was staying up. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java Changed log from ERROR to WARN and suppressed stack trace. This is the 'stop' method. It should allow that we may be going down a little unclean. No need of spew in logs. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java The tableOnMaster check in waitForMasterActive looks wrong. It was making it so a 'normal' Master was getting stuck in here. This is not the place to worry about tablesOnMaster. That is for the balancer to be concerned with. There is a problem with Master hosting system-tables-only. After further study, Master can carry regions like a regionserver but making it so it carries system tables only is tricky given meta assign happens ahead of all others which means that the Master needs to have checked-in as a regionserver super early... It needs work. Punted for now. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java Mostly renaming so lists and maps of region infos have same name as they have elsewhere in code base and cleaning up confusion that may arise when we talk of servers-for-system-tablesIt is talking about something else in the code changes here that is other than the normal understanding. It is about filtering regionservers by their version numbers so we favor regions with higher version numbers. Needs to go back up into the balancer. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java It was possible for the Master to be given regions if no regionservers available (as per the failing unit test in this case). M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Minor reordering moving the waitForMasterActive later in the initialize and wrapping each test in a check if we are to keep looping (which checks cluster status flag). M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerMetrics.java This was an old test from the days when Master carried system tables. Updated test and fixed metrics. Metrics count the hbase:meta along with the userspace region so upped expected numbers (previously the hbase:meta was hosted on the master so metrics were not incremented). M hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java I took a look at this test again but nope, needs a load of work still to make it pass. M hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java Stop being so whiney. > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-19598.master.001.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19794) TestZooKeeper hangs
[ https://issues.apache.org/jira/browse/HBASE-19794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328329#comment-16328329 ] stack commented on HBASE-19794: --- Still need to look at this. Hopefully ** HBASE-19598 is related. > TestZooKeeper hangs > --- > > Key: HBASE-19794 > URL: https://issues.apache.org/jira/browse/HBASE-19794 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-2 > > Attachments: org.apache.hadoop.hbase.TestZooKeeper-output.txt > > > Seems like the TestZKAsyncRegistry that hangs in shutdown. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[ https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328328#comment-16328328 ] stack commented on HBASE-19792: --- Looks like it does (smile). > TestReplicationSmallTests.testDisableEnable fails > - > > Key: HBASE-19792 > URL: https://issues.apache.org/jira/browse/HBASE-19792 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19792.patch, HBASE-19792.patch, HBASE-19792.patch, > org.apache.hadoop.hbase.replication.TestReplicationSmallTests-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18511) Default no regions on master
[ https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18511: -- Release Note: Changes the configuration hbase.balancer.tablesOnMaster from list of table names that the can carry (with 'none' meaning no tables on the master) to instead be a boolean that is set to true if master carries tables/regions and false if it does not. If true, the master acts like any regionserver. If false, then the master carries no tables. This is the default for hbase-2.0.0. Another boolean configuration, hbase.balancer.tablesOnMaster.systemTablesOnly, when set to true, enables hbase.balancer.tablesOnMaster and makes it so the master hosts system tables exclusively (the long-time deploy mode of master branch and branch-2 up until this commit).. UPDATE: This is broke. See HBASE-19785. The change of hbase.balancer.tablesOnMaster from String list to boolean and the addition of a simple boolean to enable system-tables on Master was done to constrain what operators might ask for via this master configuration. Stipulating what tables are bound to the Master server verges into regionserver grouping territory, a more robust means of specifying table and server combinations. Operators should use this latter if they want layouts more exotic than those supplied by the provided booleans. was: Changes the configuration hbase.balancer.tablesOnMaster from list of table names that the can carry (with 'none' meaning no tables on the master) to instead be a boolean that is set to true if master carries tables/regions and false if it does not. If true, the master acts like any regionserver. If false, then the master carries no tables. This is the default for hbase-2.0.0. Another boolean configuration, hbase.balancer.tablesOnMaster.systemTablesOnly, when set to true, enables hbase.balancer.tablesOnMaster and makes it so the master hosts system tables exclusively (the long-time deploy mode of master branch and branch-2 up until this commit). The change of hbase.balancer.tablesOnMaster from String list to boolean and the addition of a simple boolean to enable system-tables on Master was done to constrain what operators might ask for via this master configuration. Stipulating what tables are bound to the Master server verges into regionserver grouping territory, a more robust means of specifying table and server combinations. Operators should use this latter if they want layouts more exotic than those supplied by the provided booleans. > Default no regions on master > > > Key: HBASE-18511 > URL: https://issues.apache.org/jira/browse/HBASE-18511 > Project: HBase > Issue Type: Task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18511.master.001.patch, > HBASE-18511.master.002.patch, HBASE-18511.master.003.patch, > HBASE-18511.master.004.patch, HBASE-18511.master.005.patch, > HBASE-18511.master.006.patch, HBASE-18511.master.007.patch, > HBASE-18511.master.008.patch, HBASE-18511.master.009.patch, > HBASE-18511.master.010.patch, HBASE-18511.master.011.patch, > HBASE-18511.master.012.patch, HBASE-18511.master.013.patch, > HBASE-18511.master.014.patch, HBASE-18511.master.015.patch > > > Let this be umbrella issue for no-regions-on-master as default deploy (as it > was in branch-1). > Also need to make sure we can run WITH regions on master; in particular > system tables with RPC short-circuit as it is now in hbase master. > Background is that master branch carried a change that allowed Master carry > regions. On top of this improvement on branch-1, Master defaulted to carry > system tables only. No release was made with this configuration. Now we are > going to cut the 2.0.0 release, the decision is that hbase-2 should have the > same layout as hbase-1 so this issue implements the undoing of Master > carrying system tables by default (though the capability remains). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19785) System Regions on the Master is broken
[ https://issues.apache.org/jira/browse/HBASE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19785: -- Fix Version/s: 2.0.0 > System Regions on the Master is broken > -- > > Key: HBASE-19785 > URL: https://issues.apache.org/jira/browse/HBASE-19785 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > > The parent issue broke our being able to host system regions only on the > Master. > This broke a few tests that depend on this ability. Two of the below actually > enable system regions on the Master for the test run. The remainder is the > test that make sure this works. > TestMultiParallel > TestRegionsOnMasterOptions > TestRegionServerReadRequestMetrics > Parent changed the startup order. System regions and Master-as-a-RegionServer > are having issues because we wait for regionservers to check in before > completing Master startup which gets interesting when Master is supposed to > act like a RegionServer. Previously, Master startup was off in a background > thread. > Needs more thought but not required for beta-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19785) System Regions on the Master is broken
[ https://issues.apache.org/jira/browse/HBASE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328316#comment-16328316 ] stack commented on HBASE-19785: --- HBASE-19808 reenables half of the disabled tests above. I moved this out of beta-2. Unlikely to get the needed attention in time. > System Regions on the Master is broken > -- > > Key: HBASE-19785 > URL: https://issues.apache.org/jira/browse/HBASE-19785 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > > The parent issue broke our being able to host system regions only on the > Master. > This broke a few tests that depend on this ability. Two of the below actually > enable system regions on the Master for the test run. The remainder is the > test that make sure this works. > TestMultiParallel > TestRegionsOnMasterOptions > TestRegionServerReadRequestMetrics > Parent changed the startup order. System regions and Master-as-a-RegionServer > are having issues because we wait for regionservers to check in before > completing Master startup which gets interesting when Master is supposed to > act like a RegionServer. Previously, Master startup was off in a background > thread. > Needs more thought but not required for beta-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19785) System Regions on the Master is broken
[ https://issues.apache.org/jira/browse/HBASE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19785: -- Fix Version/s: (was: 2.0.0-beta-2) > System Regions on the Master is broken > -- > > Key: HBASE-19785 > URL: https://issues.apache.org/jira/browse/HBASE-19785 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > > The parent issue broke our being able to host system regions only on the > Master. > This broke a few tests that depend on this ability. Two of the below actually > enable system regions on the Master for the test run. The remainder is the > test that make sure this works. > TestMultiParallel > TestRegionsOnMasterOptions > TestRegionServerReadRequestMetrics > Parent changed the startup order. System regions and Master-as-a-RegionServer > are having issues because we wait for regionservers to check in before > completing Master startup which gets interesting when Master is supposed to > act like a RegionServer. Previously, Master startup was off in a background > thread. > Needs more thought but not required for beta-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19808) Reenable TestMultiParallel
[ https://issues.apache.org/jira/browse/HBASE-19808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19808: -- Status: Patch Available (was: Open) > Reenable TestMultiParallel > -- > > Key: HBASE-19808 > URL: https://issues.apache.org/jira/browse/HBASE-19808 > Project: HBase > Issue Type: Bug > Components: test > Environment: Reenable TestMultiParallel and half of > TestRegionServerReadRequestMetrics. They depended on Master being able to > carry the system tables exclusively. Disabling this request, they work so > just enable them again. >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: 0001-HBASE-19808-Reenable-TestMultiParallel.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19808) Reenable TestMultiParallel
[ https://issues.apache.org/jira/browse/HBASE-19808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19808: -- Attachment: 0001-HBASE-19808-Reenable-TestMultiParallel.patch > Reenable TestMultiParallel > -- > > Key: HBASE-19808 > URL: https://issues.apache.org/jira/browse/HBASE-19808 > Project: HBase > Issue Type: Bug > Components: test > Environment: Reenable TestMultiParallel and half of > TestRegionServerReadRequestMetrics. They depended on Master being able to > carry the system tables exclusively. Disabling this request, they work so > just enable them again. >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: 0001-HBASE-19808-Reenable-TestMultiParallel.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19785) System Regions on the Master is broken
[ https://issues.apache.org/jira/browse/HBASE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19785: -- Summary: System Regions on the Master is broken (was: System Regions on the Master is broken by parent HBASE-19694) > System Regions on the Master is broken > -- > > Key: HBASE-19785 > URL: https://issues.apache.org/jira/browse/HBASE-19785 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > The parent issue broke our being able to host system regions only on the > Master. > This broke a few tests that depend on this ability. Two of the below actually > enable system regions on the Master for the test run. The remainder is the > test that make sure this works. > TestMultiParallel > TestRegionsOnMasterOptions > TestRegionServerReadRequestMetrics > Parent changed the startup order. System regions and Master-as-a-RegionServer > are having issues because we wait for regionservers to check in before > completing Master startup which gets interesting when Master is supposed to > act like a RegionServer. Previously, Master startup was off in a background > thread. > Needs more thought but not required for beta-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19808) Reenable TestMultiParallel
stack created HBASE-19808: - Summary: Reenable TestMultiParallel Key: HBASE-19808 URL: https://issues.apache.org/jira/browse/HBASE-19808 Project: HBase Issue Type: Bug Components: test Environment: Reenable TestMultiParallel and half of TestRegionServerReadRequestMetrics. They depended on Master being able to carry the system tables exclusively. Disabling this request, they work so just enable them again. Reporter: stack Assignee: stack Fix For: 2.0.0-beta-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19785) System Regions on the Master is broken by parent HBASE-19694
[ https://issues.apache.org/jira/browse/HBASE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328309#comment-16328309 ] stack commented on HBASE-19785: --- I spent more time on this. The Master-as-RegionServer needs more work. The complication is special-handling assigning hbase:meta ahead of all other regions. Master would need to have checked-in as an 'ordinary' RegionServer way early in Master startup. This complicates assign. In past we had Master start up a background thread that took care of background check-in by RegionServers but it 'escaped' our control of startup sequence. What sort-of-works is that the Master can act as any other RegionServer. It'll be late to check in so will probably miss the initial assignments but should pick up regions the next time the balancer runs. TODO: backup Masters carrying regions. For Master to be true RegionServer, needs more work/refactor/thought. Meantime, I can reenable a bunch of the disabled tests above: all of TestMultiParallel if I don't stipulate system tables on Master only and half of TestRegionServerReadRequestMetrics (too lazy to figure the counts in the remainder). The TestRegionsOnMasterOptions has the three possible combinations. The system-tables-on-master only is what does not work and is disabled. > System Regions on the Master is broken by parent HBASE-19694 > > > Key: HBASE-19785 > URL: https://issues.apache.org/jira/browse/HBASE-19785 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > The parent issue broke our being able to host system regions only on the > Master. > This broke a few tests that depend on this ability. Two of the below actually > enable system regions on the Master for the test run. The remainder is the > test that make sure this works. > TestMultiParallel > TestRegionsOnMasterOptions > TestRegionServerReadRequestMetrics > Parent changed the startup order. System regions and Master-as-a-RegionServer > are having issues because we wait for regionservers to check in before > completing Master startup which gets interesting when Master is supposed to > act like a RegionServer. Previously, Master startup was off in a background > thread. > Needs more thought but not required for beta-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19807) MasterProcWALs never clean up again
jackylau created HBASE-19807: Summary: MasterProcWALs never clean up again Key: HBASE-19807 URL: https://issues.apache.org/jira/browse/HBASE-19807 Project: HBase Issue Type: Bug Components: proc-v2 Affects Versions: 1.2.0 Reporter: jackylau Fix For: 1.2.0 Attachments: Snipaste_2018-01-17_13-49-25.png, Snipaste_2018-01-17_13-51-32.png, Snipaste_2018-01-17_13-51-50.png MasterProcWALs never clean up again. There are many state logs in the directory of /hbase/MasterProcWALs 。And I narrow it down now, which you can see in the log file. >From the code, we can know three branch of file deleting logic is skip. So >there are so many file can't delete. And it creates one file every hour. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328301#comment-16328301 ] Hudson commented on HBASE-19796: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1060 (See [https://builds.apache.org/job/HBase-1.2-IT/1060/]) HBASE-19796 ReplicationSynUp tool is not replicating the data if the WAL (ashishsinghi: rev 45e99ffa68c9a7dd71173ffcb707110898950802) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.3.2, 1.2.7 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19598: -- Attachment: HBASE-19598.master.001.patch > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-19598.master.001.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328299#comment-16328299 ] Hudson commented on HBASE-19796: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #337 (See [https://builds.apache.org/job/HBase-1.3-IT/337/]) HBASE-19796 ReplicationSynUp tool is not replicating the data if the WAL (ashishsinghi: rev 04bb4082438faf87c19627a7109c714bf17113b1) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.3.2, 1.2.7 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328280#comment-16328280 ] Ashish Singhi edited comment on HBASE-19796 at 1/17/18 5:22 AM: Thanks Ted for the review. I have pushed the changes to branch-1.2 and branch-1.3 was (Author: ashish singhi): Thanks Ted for the review. > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.3.2, 1.2.7 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-19796: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 1.3.3) 1.3.2 Status: Resolved (was: Patch Available) Thanks Ted for the review. > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.3.2, 1.2.7 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328278#comment-16328278 ] Ashish Singhi commented on HBASE-19796: --- Thanks Ted. How to trigger a build manually ? Anywhere it's documented ? > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.2.7, 1.3.3 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328269#comment-16328269
]
Mike Drob commented on HBASE-19770:
---
{quote}
bq. The change is not enabling return-values by default when shell is started
in non-interactive mode.
Ahh, good catch. I forgot to do that. Agree that it should only be relevant for
interactive mode.
{quote}
V3 still didn't address this I think?
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch, HBASE-19770.003.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[
https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328267#comment-16328267
]
Hadoop QA commented on HBASE-19792:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 11m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
16s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged -
3 fixed = 1 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} hbase-mapreduce: The patch generated 0 new + 9
unchanged - 17 fixed = 9 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
21m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
32s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
14s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}176m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19792 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12906313/HBASE-19792.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6d8fed125a81 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 8b6b2b0b22 |
|
[jira] [Commented] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[
https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328258#comment-16328258
]
Hadoop QA commented on HBASE-19792:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
0s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 8m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged -
3 fixed = 1 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} hbase-mapreduce: The patch generated 0 new + 9
unchanged - 17 fixed = 9 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}106m
2s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
16s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19792 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12906313/HBASE-19792.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f3a8831044c8 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8b6b2b0b22 |
|
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328249#comment-16328249 ] Duo Zhang commented on HBASE-19803: --- https://stackoverflow.com/questions/5401281/preventing-system-exit-from-api I think we could try this? Disable System.exit when running UTs, and we can output something when System.exit is called so we can know who is the criminal. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19757) System table gets stuck after enabling region server group feature in secure cluster
[
https://issues.apache.org/jira/browse/HBASE-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328192#comment-16328192
]
Hadoop QA commented on HBASE-19757:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
6s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19757 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905897/19757.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f4ef8413276b 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8b6b2b0b22 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11075/testReport/ |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11075/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This
[jira] [Commented] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[ https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328117#comment-16328117 ] Duo Zhang commented on HBASE-19792: --- Thanks [~stack]. Let me see if the yetus pre commit admin works. > TestReplicationSmallTests.testDisableEnable fails > - > > Key: HBASE-19792 > URL: https://issues.apache.org/jira/browse/HBASE-19792 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19792.patch, HBASE-19792.patch, HBASE-19792.patch, > org.apache.hadoop.hbase.replication.TestReplicationSmallTests-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[ https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19792: -- Attachment: HBASE-19792.patch > TestReplicationSmallTests.testDisableEnable fails > - > > Key: HBASE-19792 > URL: https://issues.apache.org/jira/browse/HBASE-19792 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19792.patch, HBASE-19792.patch, HBASE-19792.patch, > org.apache.hadoop.hbase.replication.TestReplicationSmallTests-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19757) System table gets stuck after enabling region server group feature in secure cluster
[
https://issues.apache.org/jira/browse/HBASE-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328090#comment-16328090
]
Ted Yu commented on HBASE-19757:
[~apurtell]:
Can you take a look at the latest patch ?
> System table gets stuck after enabling region server group feature in secure
> cluster
>
>
> Key: HBASE-19757
> URL: https://issues.apache.org/jira/browse/HBASE-19757
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: 19757.v1.txt, 19757.v2.txt, 19757.v3.txt, 19757.v4.txt
>
>
> I was testing on an hbase-2 secure cluster against hadoop 3 where some tables
> were created without region server group feature.
> After adding the RSGroupAdminEndpoint and RSGroupBasedLoadBalancer to
> hbase-site, I restarted the whole cluster.
> After the restart, hbase:meta region got stuck in transition (forever).
> {code}
> 2018-01-10 21:20:16,696 INFO
> [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-ctr-e137-1514896590304-8706-01-02.hwx.site,2,1515619212617]
> zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at
> address=ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515618538016,
> exception=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:meta,,1 is not online on
> ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515619181453
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3314)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3291)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1355)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegionInfo(RSRpcServices.java:1667)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19739) Include thrift IDL files in HBase binary distribution
[ https://issues.apache.org/jira/browse/HBASE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328085#comment-16328085 ] Hudson commented on HBASE-19739: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4414 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4414/]) HBASE-19739 Include thrift IDL files in hbase-thrift.jar and .proto (stack: rev 8b6b2b0b227f9b16d9c85ab69afa1f968ce72400) * (edit) hbase-protocol-shaded/pom.xml * (edit) hbase-thrift/pom.xml > Include thrift IDL files in HBase binary distribution > - > > Key: HBASE-19739 > URL: https://issues.apache.org/jira/browse/HBASE-19739 > Project: HBase > Issue Type: Improvement > Components: Thrift >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19739.001.patch > > > Include thrift IDL files in HBase binary distribution, possibly at: > lib/thrift/Hbase.thrift > lib/thrift2/hbase.thrift -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19791) TestZKAsyncRegistry hangs
[ https://issues.apache.org/jira/browse/HBASE-19791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328084#comment-16328084 ] Hudson commented on HBASE-19791: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4414 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4414/]) HBASE-19791 TestZKAsyncRegistry hangs (stack: rev d3a306d81d3f087696fc6d45dd8d6bda939378b2) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestZKAsyncRegistry.java > TestZKAsyncRegistry hangs > - > > Key: HBASE-19791 > URL: https://issues.apache.org/jira/browse/HBASE-19791 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-2 > > Attachments: 0001-HBASE-19791-do-nothing.patch, jstack, output > > > It hangs in TEST_UTIL.shutdownMiniCluster() for me locally. > Will upload the test output and jstack result for further digging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328004#comment-16328004
]
Appy commented on HBASE-19770:
--
gave +1 earlier, feel free to submit whenever you're done with minor changes
afa i'm concerned. (just stating it explicitly :) )
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch, HBASE-19770.003.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327990#comment-16327990
]
Sergey Soldatov edited comment on HBASE-19805 at 1/16/18 11:27 PM:
---
well, here is a short RCA:
checkSplittable() method relies on Region.isSplittable which is just a simple
check that region is available (not closing nor not closed) and has no
references. But HRegion.closing flag we set only when we actually execute
doClose(). At first glance, it would be reasonable to add a check that the
region state (RegionStateNode) is not CLOSING to checkSplittable().
was (Author: sergey.soldatov):
well, here is a short RCA:
checkSplittable method relies on Region.isSplittable which is just a simple
check that region is available (not closing nor not closed) and has no
references. But HRegion.closing flag we set only when we actually execute
doClose(). At first glance, it would be reasonable to add a check that the
region state (RegionStateNode) is not CLOSING.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327990#comment-16327990
]
Sergey Soldatov commented on HBASE-19805:
-
well, here is a short RCA:
checkSplittable method relies on Region.isSplittable which is just a simple
check that region is available (not closing nor not closed) and has no
references. But HRegion.closing flag we set only when we actually execute
doClose(). At first glance, it would be reasonable to add a check that the
region state (RegionStateNode) is not CLOSING.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this
[jira] [Updated] (HBASE-19735) Create a minimal "client" tarball installation
[ https://issues.apache.org/jira/browse/HBASE-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19735: --- Attachment: HBASE-19735.002.branch-2.patch > Create a minimal "client" tarball installation > -- > > Key: HBASE-19735 > URL: https://issues.apache.org/jira/browse/HBASE-19735 > Project: HBase > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-19735.001.branch-2.patch, > HBASE-19735.002.branch-2.patch > > > We're moving ourselves towards more controlled dependencies. A logical next > step is to try to do the same for our "binary" artifacts that we create > during releases. > There is code (our's and our dependency's) which the HMaster and RegionServer > require which, obviously, clients do not need. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19735) Create a minimal "client" tarball installation
[ https://issues.apache.org/jira/browse/HBASE-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327965#comment-16327965 ] Josh Elser commented on HBASE-19735: I got .002 to actually do what I want it to do. Will try to combine it all into the hbase-assembly maven module next. Was able to run a basic test against a cluster as well as use the hbase shell. > Create a minimal "client" tarball installation > -- > > Key: HBASE-19735 > URL: https://issues.apache.org/jira/browse/HBASE-19735 > Project: HBase > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-19735.001.branch-2.patch, > HBASE-19735.002.branch-2.patch > > > We're moving ourselves towards more controlled dependencies. A logical next > step is to try to do the same for our "binary" artifacts that we create > during releases. > There is code (our's and our dependency's) which the HMaster and RegionServer > require which, obviously, clients do not need. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327958#comment-16327958
]
Josh Elser commented on HBASE-19770:
Just attached .003 which sets a better default value for {{return_values}} and
implements Mike's ruby-style suggestion.
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch, HBASE-19770.003.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19770:
---
Attachment: HBASE-19770.003.branch-2.patch
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch, HBASE-19770.003.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327908#comment-16327908
]
Josh Elser commented on HBASE-19805:
{quote}It should be numRegions, not newRegionSize. Otherwise this check is
valid for the first split only and that's why it run splits for the same region
without stop.
{quote}
Well, don't I have egg on my face :)
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
>
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327901#comment-16327901
]
Sergey Soldatov commented on HBASE-19805:
-
[~stack] yep, that's exactly what I've expected. Let me dig why it happen on my
own.
[~elserj] You have an error in your code:
{noformat}
Thread.sleep(50);
newRegionSize = admin.getRegions(tn).size();
{noformat}
It should be numRegions, not newRegionSize. Otherwise this check is valid for
the first split only and that's why it run splits for the same region without
stop.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There
[jira] [Commented] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327897#comment-16327897
]
Appy commented on HBASE-19780:
--
bq. With the current configuration it seems that Checkstyle doesn't get
executed anymore in the modules, where it has been enabled, when running mvn
clean install -DskipTests. I think the problem is that the goal isn't specified.
I don't think they used to run as a part of normal build earlier either. {{mvn
checkstyle:check}} takes 90 sec to build, so including checkstyle as part of
default build will substantially increase build time (from ~180sec for simple
install to ~270sec). We anyways don't need checkstyle until patch is close to
submissions and the pre-commit job already does that for us.. So in my opinion,
we should continue to exclude it from default build process. Wdys?
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327895#comment-16327895 ] Hudson commented on HBASE-19789: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #4413 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4413/]) HBASE-19789 Not exclude flaky tests from nightly builds. (appy: rev 9917244be2c7de7dccf870240f08a90baf341bbb) * (edit) dev-support/Jenkinsfile * (edit) dev-support/hbase_nightly_yetus.sh > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2 > > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19553) Old replica regions should be cleared from AM memory after primary region split or merge
[ https://issues.apache.org/jira/browse/HBASE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327891#comment-16327891 ] huaxiang sun edited comment on HBASE-19553 at 1/16/18 10:12 PM: +1, Can you fix checkStyle warning? Thanks. was (Author: huaxiang): +1, thanks. > Old replica regions should be cleared from AM memory after primary region > split or merge > > > Key: HBASE-19553 > URL: https://issues.apache.org/jira/browse/HBASE-19553 > Project: HBase > Issue Type: Bug > Components: master >Reporter: huaxiang sun >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-19553-branch-1-v2.patch, > HBASE-19553-branch-1-v3.patch, HBASE-19553-branch-1.patch > > > Similar to HBASE-18025, the replica parent's info is not removed from master. > Actually I think it can be removed after replica region is split or merged, I > will check the logic and apply one patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19553) Old replica regions should be cleared from AM memory after primary region split or merge
[ https://issues.apache.org/jira/browse/HBASE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327891#comment-16327891 ] huaxiang sun commented on HBASE-19553: -- +1, thanks. > Old replica regions should be cleared from AM memory after primary region > split or merge > > > Key: HBASE-19553 > URL: https://issues.apache.org/jira/browse/HBASE-19553 > Project: HBase > Issue Type: Bug > Components: master >Reporter: huaxiang sun >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-19553-branch-1-v2.patch, > HBASE-19553-branch-1-v3.patch, HBASE-19553-branch-1.patch > > > Similar to HBASE-18025, the replica parent's info is not removed from master. > Actually I think it can be removed after replica region is split or merged, I > will check the logic and apply one patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327887#comment-16327887
]
stack commented on HBASE-19805:
---
A region can only be split once. Something wrong if we are doing it multiple
times. FYI. I can look at log if you want to paste sir.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
> final AdminService.BlockingInterface admin =
>
>
[jira] [Created] (HBASE-19806) Lower max versions for selected system table column family
Ted Yu created HBASE-19806:
--
Summary: Lower max versions for selected system table column family
Key: HBASE-19806
URL: https://issues.apache.org/jira/browse/HBASE-19806
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
On an hbase 2 cluster, I got the description of hbase:meta table:
{code}
{NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
'FOREVER', MIN_VERSIONS => '0',
REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
'true', BLOCKSIZE => '81
92'}
...
{NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
'FOREVER', MIN_VERSIONS => '0'
, REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
'true', BLOCKSIZE => '
8192'}
{code}
You can see that 'table' family has MAX VERSIONS much higher than the other
families.
The MAX VERSIONS value should be brought in sync with the other families.
For namespace table:
{code}
{NAME => 'info', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
'FOREVER', MIN_VERSIONS => '0',
REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
'true', BLOCKSIZE => '81
92'}
{code}
Having MAX VERSIONS of 3 should be enough.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327873#comment-16327873
]
Sergey Soldatov commented on HBASE-19805:
-
[~stack] almost. Actually, that happens when one split is already in progress.
We are closing parent (so, we are setting location to null) and if at this time
we try to check whether this region is splittable, we hit this problem. I'm not
sure yet why we allow to split the same region many times. From my log
scheduled splits:
{noformat}
parent=af7ddfb3943627b825ddfe3fedb27590,
daughterA=fde4b311dd76909e05cd57f2d19a8ebc,
daughterB=4381a4dd9da46c6e7ce91ab6419fb708
parent=af7ddfb3943627b825ddfe3fedb27590,
daughterA=38d5fffbe693017be0d2fcc97eec3e3e,
daughterB=e76bfc1dddfca511c00dfd3477dc003d
parent=af7ddfb3943627b825ddfe3fedb27590,
daughterA=ef89483bc4117a31536e1c25def4f64e,
daughterB=ccf1c2f5f43f478af438b6c3f2ca7ef5
{noformat}
And only the first one is actually happen.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
>
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327858#comment-16327858 ] Appy commented on HBASE-19803: -- yeah probably, i don't see any unit test calling it (although ITs do). In the non -test code, it's mostly main() fn in tools. But digging around more: - TestZKMainServer seems to be [handling System.exit() appropriately|https://github.com/apache/hbase/blob/master/hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKMainServer.java#L85]. So do all other tests. Here's what method: https://stackoverflow.com/questions/309396/java-how-to-test-methods-that-call-system-exit - The only case where it might be wrong is, ImportTsv#createSubmittableJob calling System.exit(). TestImportTsv calls that fn multiple times. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327844#comment-16327844
]
Jan Hentschel commented on HBASE-19780:
---
Good a small point which fails the Maven build:
hbase-hadoop-compat/pom.xml => line 65 should be removed ({{}}).
With the current configuration it seems that Checkstyle doesn't get executed
anymore in the modules, where it has been enabled, when running {{mvn clean
install -DskipTests}}. I think the problem is that the {{goal}} isn't
specified. The following configuration of the plugin in the modules would fix it
{code:xml}
org.apache.maven.plugins
maven-checkstyle-plugin
checkstyle
check
true
{code}
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327774#comment-16327774 ] Appy commented on HBASE-17852: -- I'm fine with this landing in master. I'll try to take a thorough look at the code after 2.0 release (If i miss that, i'll consider myself ineligible for casting any +/- 1). Of the top of my head, I think the main areas to touch upon are: - Make backups concurrent - Use procedure framework: Long-standing request. The procv2 framework has features like locking, queuing operations, etc. Replication is already moving to it. I don't see a reason why backup can't too. - Can't use CP hooks for incremental backup. Backup should/will become first class feature - more important and critical than Coprocessor. - There should be some basic access control, if only, limiting everything to ADMIN (like RS group recently did in HBASE-19483) > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, > HBASE-17852-v3.patch, HBASE-17852-v4.patch, HBASE-17852-v5.patch, > HBASE-17852-v6.patch, HBASE-17852-v7.patch, HBASE-17852-v8.patch, > HBASE-17852-v9.patch > > > Design approach rollback-via-snapshot implemented in this ticket: > # Before backup create/delete/merge starts we take a snapshot of the backup > meta-table (backup system table). This procedure is lightweight because meta > table is small, usually should fit a single region. > # When operation fails on a server side, we handle this failure by cleaning > up partial data in backup destination, followed by restoring backup > meta-table from a snapshot. > # When operation fails on a client side (abnormal termination, for example), > next time user will try create/merge/delete he(she) will see error message, > that system is in inconsistent state and repair is required, he(she) will > need to run backup repair tool. > # To avoid multiple writers to the backup system table (backup client and > BackupObserver's) we introduce small table ONLY to keep listing of bulk > loaded files. All backup observers will work only with this new tables. The > reason: in case of a failure during backup create/delete/merge/restore, when > system performs automatic rollback, some data written by backup observers > during failed operation may be lost. This is what we try to avoid. > # Second table keeps only bulk load related references. We do not care about > consistency of this table, because bulk load is idempotent operation and can > be repeated after failure. Partially written data in second table does not > affect on BackupHFileCleaner plugin, because this data (list of bulk loaded > files) correspond to a files which have not been loaded yet successfully and, > hence - are not visible to the system -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327772#comment-16327772
]
stack commented on HBASE-19805:
---
Is it because the region is not assigned yet [~sergey.soldatov] ? Thanks.
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
> final AdminService.BlockingInterface admin =
>
> env.getMasterServices().getClusterConnection().getAdmin(regionLocation);
> {code}
> We don't
[jira] [Commented] (HBASE-19792) TestReplicationSmallTests.testDisableEnable fails
[ https://issues.apache.org/jira/browse/HBASE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327761#comment-16327761 ] stack commented on HBASE-19792: --- Skimmed RB. +1. Allen is trying to make a new yetus release to address the change in Jira behavior... > TestReplicationSmallTests.testDisableEnable fails > - > > Key: HBASE-19792 > URL: https://issues.apache.org/jira/browse/HBASE-19792 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19792.patch, HBASE-19792.patch, > org.apache.hadoop.hbase.replication.TestReplicationSmallTests-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19739) Include thrift IDL files in HBase binary distribution
[ https://issues.apache.org/jira/browse/HBASE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19739: -- Resolution: Fixed Hadoop Flags: Reviewed Release Note: Thrift IDLs are now shipped, bundled up in the respective hbase-*thrift.jars (look for files ending in .thrift). Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks [~uagashe] > Include thrift IDL files in HBase binary distribution > - > > Key: HBASE-19739 > URL: https://issues.apache.org/jira/browse/HBASE-19739 > Project: HBase > Issue Type: Improvement > Components: Thrift >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19739.001.patch > > > Include thrift IDL files in HBase binary distribution, possibly at: > lib/thrift/Hbase.thrift > lib/thrift2/hbase.thrift -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327738#comment-16327738
]
Josh Elser commented on HBASE-19805:
Oh, and a workaround is just to sleep a bit client-side :)
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
> final AdminService.BlockingInterface admin =
>
> env.getMasterServices().getClusterConnection().getAdmin(regionLocation);
> {code}
> We don't validate
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327731#comment-16327731 ] stack commented on HBASE-19803: --- I don't think any of our tests call System#exit. Would be happy if was proven wrong... > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327728#comment-16327728
]
Josh Elser commented on HBASE-19805:
Seems like a race condition from the client's view of the Regions to what the
Master's states are when the RPC makes it there.
The Client turns {{split(TableName, byte[])}} into {{splitRegion(regionId,
byte[]). }}The region ID is passed over to the master, where it performs a
lookup on that Region given the current state of the cluster. I would have to
assume that the RegionStates object from the AssignmentManager is "consistent",
so there was a change in that state.
I think the Master should just be gracefully saying "nope, can't split this"
(like what happens in HBaseAdmin for `split`)
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327715#comment-16327715
]
Josh Elser commented on HBASE-19805:
{noformat}
2018-01-16 14:07:46,054 INFO [main] client.HBaseAdmin: Started disable of
myTestTable
2018-01-16 14:07:46,930 INFO [main] client.HBaseAdmin: Operation: DISABLE,
Table Name: default:myTestTable completed
2018-01-16 14:07:47,420 INFO [main] client.HBaseAdmin: Operation: DELETE,
Table Name: default:myTestTable completed
2018-01-16 14:07:48,918 INFO [main] client.HBaseAdmin: Operation: CREATE,
Table Name: default:myTestTable completed
Splits: [1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, 10]
Splitting on 1
Splitting on 2
2018-01-16 14:07:50,159 DEBUG [main] client.ConnectionImplementation:
locateRegionInMeta parentTable=hbase:meta, metaLocation=, attempt=0 of 16
failed; retrying after sleep of 100 because: No server address listed in
hbase:meta for region
myTestTable,,1516129669054.e104c084caeff64848c0aea66a41fdc0. containing row
2018-01-16 14:07:50,267 DEBUG [main] client.ConnectionImplementation:
locateRegionInMeta parentTable=hbase:meta, metaLocation=, attempt=1 of 16
failed; retrying after sleep of 200 because: No server address listed in
hbase:meta for region
myTestTable,,1516129669054.e104c084caeff64848c0aea66a41fdc0. containing row
2018-01-16 14:07:50,473 DEBUG [main] client.ConnectionImplementation:
locateRegionInMeta parentTable=hbase:meta, metaLocation=, attempt=2 of 16
failed; retrying after sleep of 301 because: No server address listed in
hbase:meta for region
myTestTable,,1516129669054.e104c084caeff64848c0aea66a41fdc0. containing row
2018-01-16 14:07:50,785 DEBUG [main] client.ConnectionImplementation:
locateRegionInMeta parentTable=hbase:meta, metaLocation=, attempt=3 of 16
failed; retrying after sleep of 504 because: No server address listed in
hbase:meta for region
myTestTable,,1516129669054.e104c084caeff64848c0aea66a41fdc0. containing row
Splitting on 3
2018-01-16 14:07:55,828 DEBUG [main] client.RpcRetryingCallerImpl: Call
exception, tries=6, retries=6, started=4193 ms ago, cancelled=false,
msg=java.io.IOException
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:463)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
at
org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
at
org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
at
org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
... 3 more
, details=, exception=java.io.IOException: java.io.IOException
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:463)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
at
org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
at
org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
at
[jira] [Commented] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327713#comment-16327713
]
stack commented on HBASE-19780:
---
I think this patch will fix this linked issue.
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reassigned HBASE-19780:
-
Assignee: Appy
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327710#comment-16327710
]
stack commented on HBASE-19780:
---
+1 from me. Needs [~Jan Hentschel]'s blessing.
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327704#comment-16327704
]
Appy commented on HBASE-19780:
--
Ping fir review.
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327704#comment-16327704
]
Appy edited comment on HBASE-19780 at 1/16/18 8:07 PM:
---
Ping for review.
was (Author: appy):
Ping fir review.
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Priority: Major
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327693#comment-16327693
]
Mike Drob commented on HBASE-19770:
---
btw, you don't need the return keyword and you especially don't need {{return
nil}}.
Assuming, I got the condition correctly, this should work for you:
{code}
def command(command, *args)
ret = internal_command(command, :command, *args)
ret if return_values or not interactive
end
{code}
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327681#comment-16327681 ] Appy commented on HBASE-19803: -- Oh, this looks promising: http://maven.apache.org/surefire/maven-surefire-plugin/faq.html#vm-termination And we have quite a few System.exit in our code. Since the failure is in hbase-server tests, just looking for System.exit calls in that module and those on which it depends. Also, ignoring the calls from main() fns of tools. Here's list of possible culprits: - HMaster#InitializationMonitor#run() ...there may be others, but not obvious at first look. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19757) System table gets stuck after enabling region server group feature in secure cluster
[
https://issues.apache.org/jira/browse/HBASE-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327672#comment-16327672
]
Hadoop QA commented on HBASE-19757:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 37s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
15s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19757 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905897/19757.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9eaa5a52ddd6 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9917244be2 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11072/testReport/ |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11072/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327651#comment-16327651
]
Josh Elser commented on HBASE-19770:
bq. we should make this change in branch-1, branch-2, and master at least.
Maybe in 1.4.x too?
Fine for me too. Didn't spend any time thinking about where all it should land.
bq. The change is not enabling return-values by default when shell is started
in non-interactive mode.
Ahh, good catch. I forgot to do that. Agree that it should only be relevant for
interactive mode.
||Interactive||Return Values||
|True|False (default)|
|False|True (default)|
bq. nit: reverse the condition?
Sure :). I had this as {{unless return_values}} earlier and some of the ruby
checkstyle things yelled at me. Didn't reverse it entirely :P
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327642#comment-16327642 ] Ted Yu commented on HBASE-19796: Test failure was not related. > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.2.7, 1.3.3 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327631#comment-16327631
]
Josh Elser commented on HBASE-19805:
Asked Sergey if he'd like to try to take a stab at this one.
My thinking is that this should bubble up an IOException to propagate back to
the client.
FYI [~stack].
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
> final AdminService.BlockingInterface admin =
>
[jira] [Updated] (HBASE-19770) Add '--returnoption to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19770:
-
Summary: Add '--returnoption to Shell to print return values of commands in
interactive mode (was: `list` shell command does not return an array of table
names)
> Add '--returnoption to Shell to print return values of commands in
> interactive mode
> ---
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
[
https://issues.apache.org/jira/browse/HBASE-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser reassigned HBASE-19805:
--
Assignee: Sergey Soldatov
> NPE in HMaster while issuing a sequence of table splits
> ---
>
> Key: HBASE-19805
> URL: https://issues.apache.org/jira/browse/HBASE-19805
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Sergey Soldatov
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> I wrote a toy program to test the client tarball in HBASE-19735. After the
> first few region splits, I see the following error in the Master log.
> {noformat}
> 2018-01-16 14:07:52,797 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
> Client=jelser//192.168.1.23 split
> myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
> 2018-01-16 14:07:52,797 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
> at
> org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
> at
> org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
> at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {noformat}
> {code}
> public static void main(String[] args) throws Exception {
> Configuration conf = HBaseConfiguration.create();
> try (Connection conn = ConnectionFactory.createConnection(conf);
> Admin admin = conn.getAdmin()) {
> final TableName tn = TableName.valueOf("myTestTable");
> if (admin.tableExists(tn)) {
> admin.disableTable(tn);
> admin.deleteTable(tn);
> }
> final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
>
> .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
> .build();
> admin.createTable(desc);
> List splitPoints = new ArrayList<>(16);
> for (int i = 1; i <= 16; i++) {
> splitPoints.add(Integer.toString(i, 16));
> }
>
> System.out.println("Splits: " + splitPoints);
> int numRegions = admin.getRegions(tn).size();
> for (String splitPoint : splitPoints) {
> System.out.println("Splitting on " + splitPoint);
> admin.split(tn, Bytes.toBytes(splitPoint));
> Thread.sleep(200);
> int newRegionSize = admin.getRegions(tn).size();
> while (numRegions == newRegionSize) {
> Thread.sleep(50);
> newRegionSize = admin.getRegions(tn).size();
> }
> }
> {code}
> A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
> {code}
> static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
> env,
> final ServerName regionLocation, final RegionInfo hri, boolean
> includeBestSplitRow)
> throws IOException {
> // TODO: There is no timeout on this controller. Set one!
> HBaseRpcController controller =
> env.getMasterServices().getClusterConnection().
> getRpcControllerFactory().newController();
> final AdminService.BlockingInterface admin =
>
> env.getMasterServices().getClusterConnection().getAdmin(regionLocation);
> {code}
> We don't validate that we have a non-null {{ServerName regionLocation}}.
--
[jira] [Updated] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19770:
-
Summary: Add '--return-values' option to Shell to print return values of
commands in interactive mode (was: Add '--returnoption to Shell to print
return values of commands in interactive mode)
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits
Josh Elser created HBASE-19805:
--
Summary: NPE in HMaster while issuing a sequence of table splits
Key: HBASE-19805
URL: https://issues.apache.org/jira/browse/HBASE-19805
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 2.0.0-beta-1
Reporter: Josh Elser
Fix For: 2.0.0-beta-2
I wrote a toy program to test the client tarball in HBASE-19735. After the
first few region splits, I see the following error in the Master log.
{noformat}
2018-01-16 14:07:52,797 INFO
[RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster:
Client=jelser//192.168.1.23 split
myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23.
2018-01-16 14:07:52,797 ERROR
[RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer:
Unexpected throwable object
java.lang.NullPointerException
at
org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175)
at
org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149)
at
org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761)
at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618)
at
org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
{noformat}
{code}
public static void main(String[] args) throws Exception {
Configuration conf = HBaseConfiguration.create();
try (Connection conn = ConnectionFactory.createConnection(conf);
Admin admin = conn.getAdmin()) {
final TableName tn = TableName.valueOf("myTestTable");
if (admin.tableExists(tn)) {
admin.disableTable(tn);
admin.deleteTable(tn);
}
final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn)
.addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build())
.build();
admin.createTable(desc);
List splitPoints = new ArrayList<>(16);
for (int i = 1; i <= 16; i++) {
splitPoints.add(Integer.toString(i, 16));
}
System.out.println("Splits: " + splitPoints);
int numRegions = admin.getRegions(tn).size();
for (String splitPoint : splitPoints) {
System.out.println("Splitting on " + splitPoint);
admin.split(tn, Bytes.toBytes(splitPoint));
Thread.sleep(200);
int newRegionSize = admin.getRegions(tn).size();
while (numRegions == newRegionSize) {
Thread.sleep(50);
newRegionSize = admin.getRegions(tn).size();
}
}
{code}
A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame.
{code}
static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv
env,
final ServerName regionLocation, final RegionInfo hri, boolean
includeBestSplitRow)
throws IOException {
// TODO: There is no timeout on this controller. Set one!
HBaseRpcController controller =
env.getMasterServices().getClusterConnection().
getRpcControllerFactory().newController();
final AdminService.BlockingInterface admin =
env.getMasterServices().getClusterConnection().getAdmin(regionLocation);
{code}
We don't validate that we have a non-null {{ServerName regionLocation}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327622#comment-16327622
]
Appy commented on HBASE-19770:
--
Fix versions for HBASE-15965 was 1.4.0 and 2.0.0, so we should make this
change in branch-1, branch-2, and master at least. Maybe in 1.4.x too?
The change is not enabling return-values by default when shell is started in
non-interactive mode. Currently, '-n' is useless without '-r'. Shouldn't we set
return_value=true for "--noninteractive" arg case too?
nit: reverse the condition?
{noformat}
+ if not return_values
return nil
else
return ret
{noformat}
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327599#comment-16327599
]
Josh Elser commented on HBASE-19770:
bq. looks like the gist was to make the behavior of shell commands more
consistent. Some commands printed output and returned results (some printed the
results too as duplicate, some didn't), some commands didn't return any result
but just printed something, etc.
Yup, totally understand why the change was made. I think, for normal users,
this is fine. That's why I've introduced this flag (default: false) to let
those of us who *know* what we're doing to keep doing things how we've been
doing them.
bq. That said, if there's a suggestion to improve shell experience in a way
more than - print lists for some selected commands, all up for it.
It would be nice if we could just control this at the IRB level, but I haven't
looked to see if it's possible to suppress the shell printing the values.
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327598#comment-16327598
]
Appy commented on HBASE-19770:
--
Jira comments in my window were out-of-date. Your last comment makes perfect
sense [~elserj]. +1
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327593#comment-16327593
]
Appy commented on HBASE-19770:
--
Looking at the patch in other jira (HBASE-15965) again, looks like the gist was
to make the behavior of shell commands more consistent. Some commands printed
output and returned results (some printed the results too as duplicate, some
didn't), some commands didn't return any result but just printed something, etc.
It's a mess to maintain code when exceptions are norm rather than rules, what
was being suggested as alternative in that jira (i feel bad that i missed to
reply it though)
That said, if there's a suggestion to improve shell experience in a way more
than - print lists for some selected commands, all up for it.
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327574#comment-16327574 ] stack commented on HBASE-19789: --- Thanks [~appy] > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2 > > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327567#comment-16327567
]
Josh Elser edited comment on HBASE-19770 at 1/16/18 6:46 PM:
-
Thanks, Mike. No worries on another explanation.
Our [~appy] introduced the {{interactive}} flag to the shell. "Interactive"
mode was meant to be designed for when a user is sitting in front of the
terminal typing in commands, and "non-interactive" was for when a file of
commands are provided. Non-interactive mode did a few things like:
* Shell exit on first failed command in the file of commands
* Different I/O setup to IRB
* Values returned by command methods
The opposite of each of these are true for the interactive mode:
* Shell does not exit for any failed command
* Commands' return values are not given to the user
My argument is that the latter should still have the ability to "opt-in". For
example, power-users know that they're in a ruby shell and would want to do fun
things like: {{list.each\{|t| disable t; drop t\}}}. I introduced the {{-r}}
option that, regardless of the interactive/non-interactive mode, would cause
shell commands to return values to the user. The default functionality has not
changed.
was (Author: elserj):
Thanks, Mike. No worries on another explanation.
Our [~appy] introduced the {{interactive}} flag to the shell. "Interactive"
mode was meant to be designed for when a user is sitting in front of the
terminal typing in commands, and "non-interactive" was for when a file of
commands are provided. Non-interactive mode did a few things like:
* Shell exit on first failed command in the file of commands
* Different I/O setup to IRB
* Values returned by command methods
The opposite of each of these are true for the interactive mode:
* Shell does not exit for any failed command
* Commands' return values are not given to the user
My argument is that the latter should still have the ability to "opt-in". For
example, power-users know that they're in a ruby shell and would want to do fun
things like: {{\['table1', 'table2'\].each\{|t| disable t; drop t\}}}. I
introduced the {{-r}} option that, regardless of the
interactive/non-interactive mode, would cause shell commands to return values
to the user. The default functionality has not changed.
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327567#comment-16327567
]
Josh Elser commented on HBASE-19770:
Thanks, Mike. No worries on another explanation.
Our [~appy] introduced the {{interactive}} flag to the shell. "Interactive"
mode was meant to be designed for when a user is sitting in front of the
terminal typing in commands, and "non-interactive" was for when a file of
commands are provided. Non-interactive mode did a few things like:
* Shell exit on first failed command in the file of commands
* Different I/O setup to IRB
* Values returned by command methods
The opposite of each of these are true for the interactive mode:
* Shell does not exit for any failed command
* Commands' return values are not given to the user
My argument is that the latter should still have the ability to "opt-in". For
example, power-users know that they're in a ruby shell and would want to do fun
things like: {{\['table1', 'table2'\].each\{|t| disable t; drop t\}}}. I
introduced the {{-r}} option that, regardless of the
interactive/non-interactive mode, would cause shell commands to return values
to the user. The default functionality has not changed.
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19804) [hbase-indexer] Metrics source RegionServer,sub=Server already exists!
stack created HBASE-19804:
-
Summary: [hbase-indexer] Metrics source RegionServer,sub=Server
already exists!
Key: HBASE-19804
URL: https://issues.apache.org/jira/browse/HBASE-19804
Project: HBase
Issue Type: Improvement
Components: hbase-indexer
Affects Versions: 2.0.0-beta-1
Reporter: stack
Assignee: stack
Fix For: 2.0.0-beta-2
In the past, the hbase-indexer runs multiple RegionServers per JVM. In old
days, they had their own cut-down "RegionServer". In 2.0.0, we made it so they
could run an actual RegionServer but with services disabled. The latter has an
issue if you run more than one instance per JVM and it is NOT a
minihbasecluster instance. It fails with:
{code:java}
1:09:13.371 PM ERROR HRegionServer
Failed init
org.apache.hadoop.metrics2.MetricsException: Metrics source
RegionServer,sub=Server already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
at
org.apache.hadoop.hbase.metrics.BaseSourceImpl.(BaseSourceImpl.java:115)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl.(MetricsRegionServerSourceImpl.java:101)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl.(MetricsRegionServerSourceImpl.java:93)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceFactoryImpl.createServer(MetricsRegionServerSourceFactoryImpl.java:69)
at
org.apache.hadoop.hbase.regionserver.MetricsRegionServer.(MetricsRegionServer.java:56)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1519)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:954)
at com.ngdata.sep.impl.SepConsumer$1.run(SepConsumer.java:203){code}
If you look in [10:26 AM] Wolfgang Hoschek: DefaultMetricsSystem code (found by
[~whoschek]), you'll see this:
{code:java}
synchronized ObjectName newObjectName(String name) {
try {
if (mBeanNames.map.containsKey(name) && !miniClusterMode) {
throw new MetricsException(name +" already exists!");
}
return new ObjectName(mBeanNames.uniqueName(name));
} catch (Exception e) {
throw new MetricsException(e);
}
}{code}
i.e. if we are in a mini cluster context, we will not fail registering the
second bean instance.
If you look in master startup in HMasterCommandLine, you will see:
{code:java}
// If 'local', defer to LocalHBaseCluster instance. Starts master
// and regionserver both in the one JVM.
if (LocalHBaseCluster.isLocal(conf)) {
DefaultMetricsSystem.setMiniClusterMode(true);
{code}
... will ensure we don't get the above exception in minihbasecluster context.
So, the idea here is to make it so being able to run more than one RS per JVM
is cleaner than doing the above hack. It needs to be a config too a config.
which says don't fail startup if second mbean registration just because two RS
in the one context (A later issue will be the accounting of metrics per RS...
If more than one RS, then we should make a unique mbean per RS in the JVM).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19757) System table gets stuck after enabling region server group feature in secure cluster
[
https://issues.apache.org/jira/browse/HBASE-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19757:
---
Priority: Critical (was: Major)
Fix Version/s: 2.0.0-beta-2
> System table gets stuck after enabling region server group feature in secure
> cluster
>
>
> Key: HBASE-19757
> URL: https://issues.apache.org/jira/browse/HBASE-19757
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: 19757.v1.txt, 19757.v2.txt, 19757.v3.txt, 19757.v4.txt
>
>
> I was testing on an hbase-2 secure cluster against hadoop 3 where some tables
> were created without region server group feature.
> After adding the RSGroupAdminEndpoint and RSGroupBasedLoadBalancer to
> hbase-site, I restarted the whole cluster.
> After the restart, hbase:meta region got stuck in transition (forever).
> {code}
> 2018-01-10 21:20:16,696 INFO
> [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-ctr-e137-1514896590304-8706-01-02.hwx.site,2,1515619212617]
> zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at
> address=ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515618538016,
> exception=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:meta,,1 is not online on
> ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515619181453
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3314)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3291)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1355)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegionInfo(RSRpcServices.java:1667)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327544#comment-16327544 ] Appy commented on HBASE-19803: -- But the [console output|https://builds.apache.org/job/HBASE-Flaky-Tests/24830/consoleFull] doesn't say anything about out-of-memory, which i have seen in some cases in the past. So maybe it's not the issue. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327543#comment-16327543 ] Hudson commented on HBASE-19789: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1059 (See [https://builds.apache.org/job/HBase-1.2-IT/1059/]) HBASE-19789 Not exclude flaky tests from nightly builds. (appy: rev 1a97b33e1cde9b534157a555c8799ee6911a8246) * (edit) dev-support/hbase_nightly_yetus.sh * (edit) dev-support/Jenkinsfile > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2 > > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327540#comment-16327540 ] Hudson commented on HBASE-19789: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #336 (See [https://builds.apache.org/job/HBase-1.3-IT/336/]) HBASE-19789 Not exclude flaky tests from nightly builds. (appy: rev 28f811420368450ba12be77c2836d529938b2d75) * (edit) dev-support/hbase_nightly_yetus.sh * (edit) dev-support/Jenkinsfile > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2 > > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19757) System table gets stuck after enabling region server group feature in secure cluster
[
https://issues.apache.org/jira/browse/HBASE-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327527#comment-16327527
]
Ted Yu commented on HBASE-19757:
Since we cannot rely on masterServices to send back the list of system tables,
in patch v4, I aligned the system table detection code with branch-1.
> System table gets stuck after enabling region server group feature in secure
> cluster
>
>
> Key: HBASE-19757
> URL: https://issues.apache.org/jira/browse/HBASE-19757
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19757.v1.txt, 19757.v2.txt, 19757.v3.txt, 19757.v4.txt
>
>
> I was testing on an hbase-2 secure cluster against hadoop 3 where some tables
> were created without region server group feature.
> After adding the RSGroupAdminEndpoint and RSGroupBasedLoadBalancer to
> hbase-site, I restarted the whole cluster.
> After the restart, hbase:meta region got stuck in transition (forever).
> {code}
> 2018-01-10 21:20:16,696 INFO
> [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-ctr-e137-1514896590304-8706-01-02.hwx.site,2,1515619212617]
> zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at
> address=ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515618538016,
> exception=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:meta,,1 is not online on
> ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515619181453
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3314)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3291)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1355)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegionInfo(RSRpcServices.java:1667)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327526#comment-16327526 ] Appy commented on HBASE-19803: -- bq. It seems that the jvm may crash during the mvn test run and then we will kill all the running tests and then we may mark some of them as hang which leads to the false positive. Makes sense. This one suggests that it can be memory issue : https://stackoverflow.com/questions/42298883/maven-build-failure-when-running-tests-due-to-jvm-crash Looking at old nighly job (https://builds.apache.org/job/HBase-Trunk_matrix/configure), it was using -Xmx6100M. But the new jobs seem to be using just 3g (https://github.com/apache/hbase/blob/master/dev-support/docker/Dockerfile#L40) > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19757) System table gets stuck after enabling region server group feature in secure cluster
[
https://issues.apache.org/jira/browse/HBASE-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19757:
---
Attachment: 19757.v4.txt
> System table gets stuck after enabling region server group feature in secure
> cluster
>
>
> Key: HBASE-19757
> URL: https://issues.apache.org/jira/browse/HBASE-19757
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19757.v1.txt, 19757.v2.txt, 19757.v3.txt, 19757.v4.txt
>
>
> I was testing on an hbase-2 secure cluster against hadoop 3 where some tables
> were created without region server group feature.
> After adding the RSGroupAdminEndpoint and RSGroupBasedLoadBalancer to
> hbase-site, I restarted the whole cluster.
> After the restart, hbase:meta region got stuck in transition (forever).
> {code}
> 2018-01-10 21:20:16,696 INFO
> [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-ctr-e137-1514896590304-8706-01-02.hwx.site,2,1515619212617]
> zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at
> address=ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515618538016,
> exception=org.apache.hadoop.hbase.NotServingRegionException:
> hbase:meta,,1 is not online on
> ctr-e137-1514896590304-8706-01-04.hwx.site,16020,1515619181453
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3314)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3291)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1355)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegionInfo(RSRpcServices.java:1667)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327505#comment-16327505
]
Mike Drob commented on HBASE-19770:
---
A bit confused by how the problem relates to the solution here... Can you give
it another go at explaining please, [~elserj]? The problem of not having exit
status seems clear enough to solve on it's own, but I'm not connecting the dots
on how this relates to array passing.
Also, want to make sure I understand the impact - we currently have scripts
that use {{-n}} and check the exit status. Will they need to be updated to use
{{-r}} as well? I think non-interactive mode does always want it, and
interactive mode can be a toggle, that's fine.
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19789. -- Resolution: Fixed Fix Version/s: 2.0.0-beta-2 1.2.7 1.5.0 1.4.1 1.3.2 > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2 > > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19789: - Summary: Not exclude flaky tests from nightly builds (was: Exclude flaky tests from nightly branch-2 runs) > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19789) Exclude flaky tests from nightly branch-2 runs
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327490#comment-16327490 ] Appy commented on HBASE-19789: -- We have been excluding flakies from nightly build since the start. The idea was, green nightly builds will 1) signify that the list of flakies is complete, 2) make people 'not-used-to' red builds. But second one seems like shallow now, given we never really focused to fix the tests from the flaky list either. I don't mind changing to keep running all tests in nightly (and not excluding flakies). Let me push the patch for that. > Exclude flaky tests from nightly branch-2 runs > -- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[
https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327432#comment-16327432
]
Hadoop QA commented on HBASE-19796:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
43s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
0s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
45s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 7s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 90m 12s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestEndToEndSplitTransaction |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-19796 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splitting directory
[ https://issues.apache.org/jira/browse/HBASE-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327295#comment-16327295 ] Ted Yu commented on HBASE-19796: Triggered QA run: https://builds.apache.org/job/PreCommit-HBASE-Build/11071/ > ReplicationSynUp tool is not replicating the data if the WAL is moved to > splitting directory > > > Key: HBASE-19796 > URL: https://issues.apache.org/jira/browse/HBASE-19796 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1 >Reporter: Ashish Singhi >Assignee: Ashish Singhi >Priority: Major > Fix For: 1.2.7, 1.3.3 > > Attachments: HBASE-19796.branch-1.3.patch, > HBASE-19796.branch-1.3.patch > > > In our test cluster we found that ReplictionSyncUp tool is not replicating > the data from the source cluster RS WAL which is moved to WAL splitting > directory to the peer cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[
https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327268#comment-16327268
]
Balazs Meszaros commented on HBASE-19598:
-
I wrote the simplest test case which can demonstrate the issue (TestUtil.java
attachment). If we abort a region server, then
{{TEST_UTIL.shutdownMiniCluster()}} hangs forever.
> Fix TestAssignmentManagerMetrics flaky test
> ---
>
> Key: HBASE-19598
> URL: https://issues.apache.org/jira/browse/HBASE-19598
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Attachments: TestUtil.java
>
>
> TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that
> commit 010012cbcb broke it (HBASE-18946).
> The test method runs successfully, but it cannot shut the minicluster down,
> and hangs forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-19598: Attachment: TestUtil.java > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327262#comment-16327262
]
Josh Elser commented on HBASE-19770:
[~mdrob], do you have time for a quick review here?
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch,
> HBASE-19770.002.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19800) hbase shell api 'list' or program api prefix has regex problem
[
https://issues.apache.org/jira/browse/HBASE-19800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19800:
---
Description:
while using list command in hbase shell, most of all works well except which
one contains 'd' char, as well as hbase program api prefix regex.
eg. list 'd.\*' wont work, but '^d\[\da-f\]\{31\}' works well. and 'd.\*'
performs just listing all of the tables.
!image-2018-01-16-12-13-25-723.png!
was:
while using list command in hbase shell, most of all works well except which
one contains 'd' char, as well as hbase program api prefix regex.
eg. list 'd.*' wont work, but '^d\[\da-f\]\{31\}' works well. and 'd.*'
performs just listing all of the tables.
!image-2018-01-16-12-13-25-723.png!
> hbase shell api 'list' or program api prefix has regex problem
> --
>
> Key: HBASE-19800
> URL: https://issues.apache.org/jira/browse/HBASE-19800
> Project: HBase
> Issue Type: Bug
> Components: API, shell
>Affects Versions: 1.2.1
> Environment: hadoop 2.7.3
> hbase 1.2.1
>Reporter: zzzhy
>Priority: Major
> Attachments: image-2018-01-16-12-08-20-327.png,
> image-2018-01-16-12-13-25-723.png
>
>
> while using list command in hbase shell, most of all works well except which
> one contains 'd' char, as well as hbase program api prefix regex.
> eg. list 'd.\*' wont work, but '^d\[\da-f\]\{31\}' works well. and 'd.\*'
> performs just listing all of the tables.
> !image-2018-01-16-12-13-25-723.png!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19800) hbase shell api 'list' or program api prefix has regex problem
[
https://issues.apache.org/jira/browse/HBASE-19800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19800:
---
Description:
while using list command in hbase shell, most of all works well except which
one contains 'd' char, as well as hbase program api prefix regex.
eg. list 'd.*' wont work, but '^d[\da-f]{31}' works well. and 'd.*' performs
just listing all of the tables.
!image-2018-01-16-12-13-25-723.png!
was:
while using list command in hbase shell, most of all works well except which
one contains 'd' char, as well as hbase program api prefix regex.
eg. list 'd.*' wont work, but '^d[\da-f]\{31}' works well. and 'd.*' performs
just listing all of the tables.
!image-2018-01-16-12-13-25-723.png!
> hbase shell api 'list' or program api prefix has regex problem
> --
>
> Key: HBASE-19800
> URL: https://issues.apache.org/jira/browse/HBASE-19800
> Project: HBase
> Issue Type: Bug
> Components: API, shell
>Affects Versions: 1.2.1
> Environment: hadoop 2.7.3
> hbase 1.2.1
>Reporter: zzzhy
>Priority: Major
> Attachments: image-2018-01-16-12-08-20-327.png,
> image-2018-01-16-12-13-25-723.png
>
>
> while using list command in hbase shell, most of all works well except which
> one contains 'd' char, as well as hbase program api prefix regex.
> eg. list 'd.*' wont work, but '^d[\da-f]{31}' works well. and 'd.*' performs
> just listing all of the tables.
> !image-2018-01-16-12-13-25-723.png!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
