[jira] [Commented] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358125#comment-16358125 ] Duo Zhang commented on HBASE-19964: --- OK I think this is something happened before. Output the memstore size after the compaction check, it is not zero. This means that we fail to flush the table. Change to use HBaseTestingUtility.flush can fix the problem. But anyway, I'm still a little worried about the Admin.flush, we may fail to flush without any notification to users. > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Reporter: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[ https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358144#comment-16358144 ] Guanghao Zhang commented on HBASE-19923: [~Apache9] Any more comments? > Reset peer state and config when refresh replication source failed > -- > > Key: HBASE-19923 > URL: https://issues.apache.org/jira/browse/HBASE-19923 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19923.master.001.patch, > HBASE-19923.master.001.patch, HBASE-19923.master.002.patch, > HBASE-19923.master.003.patch > > > Now we use procedure for replication. When peer state changed, the RS will > read peer state from storage to cache. If RS found the peer state changed, > then it will refresh replication source. If refresh failed, the Master will > retry the procedure. Then RS will read peer state again, but now the peer > state in cache is right. So it don't refresh replication source.. So we > need reset the peer state to old peer state when refresh failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[ https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358146#comment-16358146 ] Duo Zhang commented on HBASE-19923: --- No. +1. > Reset peer state and config when refresh replication source failed > -- > > Key: HBASE-19923 > URL: https://issues.apache.org/jira/browse/HBASE-19923 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19923.master.001.patch, > HBASE-19923.master.001.patch, HBASE-19923.master.002.patch, > HBASE-19923.master.003.patch > > > Now we use procedure for replication. When peer state changed, the RS will > read peer state from storage to cache. If RS found the peer state changed, > then it will refresh replication source. If refresh failed, the Master will > retry the procedure. Then RS will read peer state again, but now the peer > state in cache is right. So it don't refresh replication source.. So we > need reset the peer state to old peer state when refresh failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19964: -- Attachment: HBASE-19964.patch > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Reporter: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19964: -- Component/s: (was: Coprocessors) test > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19964: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19957) General framework to transit sync replication state
[
https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358154#comment-16358154
]
Hadoop QA commented on HBASE-19957:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
26s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
27s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
7s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} hbase-replication: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} hbase-server: The patch generated 2 new + 346 unchanged
- 0 fixed = 348 total (was 346) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 8s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
11s{color} | {color:red} hbase-replication generated 2 new + 0 unchanged - 0
fixed = 2 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
15s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
50s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 38s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
25s{color} | {color:green} The patch does not generate ASF Lic
[jira] [Commented] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[
https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358156#comment-16358156
]
Guanghao Zhang commented on HBASE-19965:
When split a region, the parent region was marked as State.SPLIT. And the
daughter regions were marked as State.SPLITTING_NEW. But for merge regions,
only the new child region was marked State.MERGED. And don't change the parent
regions state.
{code:java}
public void markRegionAsSplit(final RegionInfo parent, final ServerName
serverName,
final RegionInfo daughterA, final RegionInfo daughterB)
throws IOException {
// Update hbase:meta. Parent will be marked offline and split up in
hbase:meta.
// The parent stays in regionStates until cleared when removed by
CatalogJanitor.
// Update its state in regionStates to it shows as offline and split when
read
// later figuring what regions are in a table and what are not: see
// regionStates#getRegionsOfTable
final RegionStateNode node =
regionStates.getOrCreateRegionStateNode(parent);
node.setState(State.SPLIT);
final RegionStateNode nodeA =
regionStates.getOrCreateRegionStateNode(daughterA);
nodeA.setState(State.SPLITTING_NEW);
final RegionStateNode nodeB =
regionStates.getOrCreateRegionStateNode(daughterB);
nodeB.setState(State.SPLITTING_NEW);
regionStateStore.splitRegion(parent, daughterA, daughterB, serverName);
if (shouldAssignFavoredNodes(parent)) {
List onlineServers =
this.master.getServerManager().getOnlineServersList();
((FavoredNodesPromoter)getBalancer()).
generateFavoredNodesForDaughter(onlineServers, parent, daughterA,
daughterB);
}
}
/**
* When called here, the merge has happened. The two merged regions have been
* unassigned and the above markRegionClosed has been called on each so they
have been
* disassociated from a hosting Server. The merged region will be open after
this call. The
* merged regions are removed from hbase:meta below> Later they are deleted
from the filesystem
* by the catalog janitor running against hbase:meta. It notices when the
merged region no
* longer holds references to the old regions.
*/
public void markRegionAsMerged(final RegionInfo child, final ServerName
serverName,
final RegionInfo mother, final RegionInfo father) throws IOException {
final RegionStateNode node = regionStates.getOrCreateRegionStateNode(child);
node.setState(State.MERGED);
regionStates.deleteRegion(mother);
regionStates.deleteRegion(father);
regionStateStore.mergeRegions(child, mother, father, serverName);
if (shouldAssignFavoredNodes(child)) {
((FavoredNodesPromoter)getBalancer()).
generateFavoredNodesForMergedRegion(child, mother, father);
}
}
{code}
> Fix flaky TestAsyncRegionAdminApi
> -
>
> Key: HBASE-19965
> URL: https://issues.apache.org/jira/browse/HBASE-19965
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> See
> [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/]
>
> java.lang.AssertionError: expected:<2> but was:<3> at
> org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359)
>
> Merge regions not work. The table still have 3 regions after the
> MergeRegionsProcedure finished.
> The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because
> the MergeRegionsProcedure pid=138 start work first, so the balance need wait
> for the lock. But after merge regions finished, the MoveRegionProcedure
> pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new
> region server. This is not right. The MoveRegionProcedure should skip to
> assign a region which was marked as offline. Or we should clear the merged
> regions' procedure when MergeRegionsProcedure finished.
>
> Logs:
> 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1]
> master.HMaster(1454): balance
> hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.,
> source=cd4730e3eae2,39077,1518106776411,
> destination=cd4730e3eae2,40578,1518106776318
> 2018-02-08 16:24:44,608 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885]
> procedure2.ProcedureExecutor(868): Stored pid=138,
> state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure
> table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4,
> 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false
> ..
> 2018-02-08 16:24:50,111 INFO [PEWorker-13]
> procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS;
> MergeTableRegionsProcedure table=testMergeRegions,
> regions=[9e2773ba1efba79a
[jira] [Commented] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[
https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358190#comment-16358190
]
Hadoop QA commented on HBASE-19964:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
17s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19964 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909916/HBASE-19964.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 232d0ccfcea6 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bc720374c3 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11465/testReport/ |
| Max. process+thread count | 2387 (vs. ulimit of 1) |
| modules | C: hbase-examples U: hbase-examples |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11465/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> TestWriteHeavyIncrementObserver fails
> -
>
> Key: HBASE-19964
> URL: https://issues
[jira] [Commented] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358192#comment-16358192 ] Duo Zhang commented on HBASE-19964: --- Let me commit. > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358203#comment-16358203 ] Duo Zhang commented on HBASE-19964: --- Pushed to master and branch-2. Keep it open for a while to see if it really works. > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19957) General framework to transit sync replication state
[ https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19957: -- Attachment: HBASE-19957-HBASE-19064-v1.patch > General framework to transit sync replication state > --- > > Key: HBASE-19957 > URL: https://issues.apache.org/jira/browse/HBASE-19957 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-19957-HBASE-19064-v1.patch, > HBASE-19957-HBASE-19064.patch, HBASE-19957.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19957) General framework to transit sync replication state
[ https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358217#comment-16358217 ] Duo Zhang commented on HBASE-19957: --- Address the comments on rb. And also fix the issues found by pre commit. > General framework to transit sync replication state > --- > > Key: HBASE-19957 > URL: https://issues.apache.org/jira/browse/HBASE-19957 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-19957-HBASE-19064-v1.patch, > HBASE-19957-HBASE-19064.patch, HBASE-19957.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[
https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358221#comment-16358221
]
Guanghao Zhang commented on HBASE-19965:
The impl seems not same with the javadoc..
{code:java}
public enum State {
OFFLINE,// region is in an offline state
OPENING,// server has begun to open but not yet done
OPEN, // server opened region and updated meta
CLOSING,// server has begun to close but not yet done
CLOSED, // server closed region and updated meta
SPLITTING, // server started split of a region
SPLIT, // server completed split of a region
FAILED_OPEN,// failed to open, and won't retry any more
FAILED_CLOSE, // failed to close, and won't retry any more
MERGING,// server started merge a region
MERGED, // server completed merge a region
SPLITTING_NEW, // new region to be created when RS splits a parent
// region but hasn't be created yet, or master doesn't
// know it's already created
MERGING_NEW;// new region to be created when RS merges two
// daughter regions but hasn't be created yet, or
// master doesn't know it's already created
{code}
> Fix flaky TestAsyncRegionAdminApi
> -
>
> Key: HBASE-19965
> URL: https://issues.apache.org/jira/browse/HBASE-19965
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
> See
> [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/]
>
> java.lang.AssertionError: expected:<2> but was:<3> at
> org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359)
>
> Merge regions not work. The table still have 3 regions after the
> MergeRegionsProcedure finished.
> The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because
> the MergeRegionsProcedure pid=138 start work first, so the balance need wait
> for the lock. But after merge regions finished, the MoveRegionProcedure
> pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new
> region server. This is not right. The MoveRegionProcedure should skip to
> assign a region which was marked as offline. Or we should clear the merged
> regions' procedure when MergeRegionsProcedure finished.
>
> Logs:
> 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1]
> master.HMaster(1454): balance
> hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.,
> source=cd4730e3eae2,39077,1518106776411,
> destination=cd4730e3eae2,40578,1518106776318
> 2018-02-08 16:24:44,608 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885]
> procedure2.ProcedureExecutor(868): Stored pid=138,
> state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure
> table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4,
> 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false
> ..
> 2018-02-08 16:24:50,111 INFO [PEWorker-13]
> procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS;
> MergeTableRegionsProcedure table=testMergeRegions,
> regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479],
> forcibly=false in 5.5710sec
> 2018-02-08 16:24:50,113 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler(813): pid=139,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.,
> source=cd4730e3eae2,39077,1518106776411,
> destination=cd4730e3eae2,40578,1518106776318 testMergeRegions
> testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[ https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358279#comment-16358279 ] Guanghao Zhang commented on HBASE-19923: Pushed to master and HBASE-19397-branch-2. Thanks [~Apache9] for reviewing. > Reset peer state and config when refresh replication source failed > -- > > Key: HBASE-19923 > URL: https://issues.apache.org/jira/browse/HBASE-19923 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19923.master.001.patch, > HBASE-19923.master.001.patch, HBASE-19923.master.002.patch, > HBASE-19923.master.003.patch > > > Now we use procedure for replication. When peer state changed, the RS will > read peer state from storage to cache. If RS found the peer state changed, > then it will refresh replication source. If refresh failed, the Master will > retry the procedure. Then RS will read peer state again, but now the peer > state in cache is right. So it don't refresh replication source.. So we > need reset the peer state to old peer state when refresh failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19554) AbstractTestDLS.testThreeRSAbort sometimes fails in pre commit
[
https://issues.apache.org/jira/browse/HBASE-19554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358324#comment-16358324
]
Duo Zhang commented on HBASE-19554:
---
I guess the problem is related to meta reassign. [~zghaobac] has already found
something strange for AMv2 in HBASE-19965, so wait for his digging result.
And I found something strange in the output
{noformat}
java.lang.AssertionError
at
org.apache.hadoop.hbase.wal.WALKeyImpl.getWriteEntry(WALKeyImpl.java:82)
at
org.apache.hadoop.hbase.regionserver.wal.WALUtil.doFullAppendTransaction(WALUtil.java:159)
at
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:132)
at
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:97)
at
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionCloseMarker(HRegion.java:1103)
at
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1615)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1437)
at
org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:104)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The assertion is
{code}
public MultiVersionConcurrencyControl.WriteEntry getWriteEntry() throws
InterruptedIOException {
assert this.writeEntry != null;
return this.writeEntry;
}
{code}
I think the problem is introduced by HBASE-19929. Since we have closed WAL
directly, then it is possible that we fail a WAL.append without assigning a
mvcc writeEntry. Let me file a issue to fix this.
> AbstractTestDLS.testThreeRSAbort sometimes fails in pre commit
> --
>
> Key: HBASE-19554
> URL: https://issues.apache.org/jira/browse/HBASE-19554
> Project: HBase
> Issue Type: Sub-task
> Components: Recovery, wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19554.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/10554/artifact/patchprocess/patch-unit-hbase-server.txt
> The error message is a bit strange:
> {quote}
> [ERROR] testThreeRSAbort(org.apache.hadoop.hbase.master.TestDLSAsyncFSWAL)
> Time elapsed: 20.627 s <<< ERROR!
> org.apache.hadoop.hbase.TableNotFoundException: Region of
> 'hbase:namespace,,1513320505933.451650152885a3b41d0b1110deca513c.' is
> expected in the table of 'testThreeRSAbort', but hbase:meta says it is in the
> table of 'hbase:namespace'. hbase:meta might be damaged.
> {quote}
> It fails for both FSHLog and AsyncFSWAL. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
Duo Zhang created HBASE-19966: - Summary: The WriteEntry for WALKey maybe null if we failed to call WAL.append Key: HBASE-19966 URL: https://issues.apache.org/jira/browse/HBASE-19966 Project: HBase Issue Type: Bug Components: wal Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.0.0-beta-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19966: -- Attachment: HBASE-19966.patch > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19966: -- Status: Patch Available (was: Open) > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358349#comment-16358349 ] Duo Zhang commented on HBASE-19966: --- Remove the assertion and add a null check in WALUtil.doFullAppendTransaction. Add a test to confirm that the WriteEntry could be null if we append on a closed WAL. Also cleanup some warnings for the tests. > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19957) General framework to transit sync replication state
[
https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358366#comment-16358366
]
Hadoop QA commented on HBASE-19957:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
26s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
15s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
57s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 2m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19957 |
| JIRA Patch URL |
https://iss
[jira] [Commented] (HBASE-19957) General framework to transit sync replication state
[ https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358373#comment-16358373 ] Duo Zhang commented on HBASE-19957: --- Any other concerns? [~zghaobac]? If no let's commit this and start working on the follow-on issues. This one has been blocking us for a long time... Thanks. > General framework to transit sync replication state > --- > > Key: HBASE-19957 > URL: https://issues.apache.org/jira/browse/HBASE-19957 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-19957-HBASE-19064-v1.patch, > HBASE-19957-HBASE-19064.patch, HBASE-19957.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358397#comment-16358397 ] Hudson commented on HBASE-19942: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4554 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4554/]) HBASE-19942 (addendum) Fix flaky TestSimpleRpcScheduler (zghao: rev 0a5f26324cfe4eb39eb48947f11364ec39221fa6) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19942.branch-2.001.patch, > HBASE-19942.branch-2.addendum.patch, HBASE-19942.master.001.patch, > HBASE-19942.master.addendum.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized
[
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358398#comment-16358398
]
Hudson commented on HBASE-19937:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4554 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4554/])
HBASE-19937 Ensure createRSGroupTable be called after ProcedureExecutor (tedyu:
rev bc720374c33701c13f6227c0edfe3a792e7aa333)
* (edit)
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManager.java
* (add)
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroup.java
* (edit)
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
* (edit)
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer
> are initialized
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19937.branch-2.001.patch,
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch,
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch,
> HBASE-19937.branch-2.006.patch, import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master
> load system coprocessors in
>
> {code:java}
> 844 this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it
> will encounter Exception as follows,
>
> {noformat}
> Exception in thread
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
> java.lang.IllegalArgumentException
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor initialized by calling
> {code:java}
> 848 startServiceThreads();{code}
> And LoadBalancer is initialized by calling
> {code:java}
> 868 this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS;
> CreateTableProcedure table=hbase:rsgroup
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>
> As a result of CreateTableProcedure.rollbackState, it may then print logs
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN
> org.apache.hadoop.hbase.rsgroup.RSGroupInf
[jira] [Commented] (HBASE-19957) General framework to transit sync replication state
[ https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358406#comment-16358406 ] Guanghao Zhang commented on HBASE-19957: +1 > General framework to transit sync replication state > --- > > Key: HBASE-19957 > URL: https://issues.apache.org/jira/browse/HBASE-19957 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-19957-HBASE-19064-v1.patch, > HBASE-19957-HBASE-19064.patch, HBASE-19957.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19957) General framework to transit sync replication state
[ https://issues.apache.org/jira/browse/HBASE-19957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19957: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HBASE-19064 Status: Resolved (was: Patch Available) Rebased and pushed to branch HBASE-19064. Thanks [~zghaobac] for reviewing. > General framework to transit sync replication state > --- > > Key: HBASE-19957 > URL: https://issues.apache.org/jira/browse/HBASE-19957 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: HBASE-19064 > > Attachments: HBASE-19957-HBASE-19064-v1.patch, > HBASE-19957-HBASE-19064.patch, HBASE-19957.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19082) Reject read/write from client but accept write from replication in state S
[ https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19082: -- Summary: Reject read/write from client but accept write from replication in state S (was: Implement a procedure to convert RS from DA to S) > Reject read/write from client but accept write from replication in state S > -- > > Key: HBASE-19082 > URL: https://issues.apache.org/jira/browse/HBASE-19082 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Priority: Major > Labels: HBASE-19064 > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19082) Reject read/write from client but accept write from replication in state S
[ https://issues.apache.org/jira/browse/HBASE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358482#comment-16358482 ] Duo Zhang commented on HBASE-19082: --- We use the same procedure for transiting so changed the title to be more specific. > Reject read/write from client but accept write from replication in state S > -- > > Key: HBASE-19082 > URL: https://issues.apache.org/jira/browse/HBASE-19082 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Duo Zhang >Priority: Major > Labels: HBASE-19064 > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[
https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358553#comment-16358553
]
Hadoop QA commented on HBASE-19966:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
11s{color} | {color:red} hbase-server: The patch generated 3 new + 13 unchanged
- 6 fixed = 16 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 20s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestReplicationDroppedTables |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19966 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909936/HBASE-19966.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 35b1b7260894 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 2b63af376e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11467/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11467/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11467/testReport/ |
| Max. process+thread count | 4704 (vs. ulimit of 10
[jira] [Resolved] (HBASE-4128) Detect whether there was zookeeper ensemble, master or region server hanging from previous build
[ https://issues.apache.org/jira/browse/HBASE-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-4128. --- Resolution: Not A Problem > Detect whether there was zookeeper ensemble, master or region server hanging > from previous build > > > Key: HBASE-4128 > URL: https://issues.apache.org/jira/browse/HBASE-4128 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Priority: Major > > Quite often, we see unit test(s) time out after 15 minutes. > One example was TestShell: > https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/239/console > This may be caused by zookeeper ensemble, master or region server hanging > from previous build. > We should detect (and terminate, if possible) the hanging zk ensemble, master > or region server from previous build as the first step in current build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-19920:
--
Status: Open (was: Patch Available)
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-19920:
--
Status: Patch Available (was: Open)
> TokenUtil.obtainToken unnecessarily creates a local directory
> -
>
> Key: HBASE-19920
> URL: https://issues.apache.org/jira/browse/HBASE-19920
> Project: HBase
> Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0
>
> Attachments: HBASE-19920.patch, HBASE-19920.v2.patch,
> HBASE-19920.v3.patch, HBASE-19920.v4.patch, HBASE-19920.v5.patch,
> HBASE-19920.v6.patch, HBASE-19920.v7.patch, HBASE-19920.v8.patch
>
>
> On client code, when one calls TokenUtil.obtainToken it loads ProtobufUtil
> which in its static block initializes DynamicClassLoader and that creates the
> directory ${hbase.local.dir}/jars/ and also instantiates a filesystem class
> to access hbase.dynamic.jars.dir.
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/DynamicClassLoader.java#L109-L127
> Since this is region server specific code, not expecting this to happen when
> one accesses hbase as a client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[ https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-19965: - Assignee: stack > Fix flaky TestAsyncRegionAdminApi > - > > Key: HBASE-19965 > URL: https://issues.apache.org/jira/browse/HBASE-19965 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: stack >Priority: Major > > See > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/] > > java.lang.AssertionError: expected:<2> but was:<3> at > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359) > > Merge regions not work. The table still have 3 regions after the > MergeRegionsProcedure finished. > The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because > the MergeRegionsProcedure pid=138 start work first, so the balance need wait > for the lock. But after merge regions finished, the MoveRegionProcedure > pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new > region server. This is not right. The MoveRegionProcedure should skip to > assign a region which was marked as offline. Or we should clear the merged > regions' procedure when MergeRegionsProcedure finished. > > Logs: > 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1] > master.HMaster(1454): balance > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 > 2018-02-08 16:24:44,608 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885] > procedure2.ProcedureExecutor(868): Stored pid=138, > state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure > table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4, > 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false > .. > 2018-02-08 16:24:50,111 INFO [PEWorker-13] > procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS; > MergeTableRegionsProcedure table=testMergeRegions, > regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479], > forcibly=false in 5.5710sec > 2018-02-08 16:24:50,113 INFO [PEWorker-13] > procedure.MasterProcedureScheduler(813): pid=139, > state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 testMergeRegions > testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[ https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358602#comment-16358602 ] stack commented on HBASE-19965: --- Thank you for the research [~zghaobac]; let me have a go at it boss. Nice work sir. > Fix flaky TestAsyncRegionAdminApi > - > > Key: HBASE-19965 > URL: https://issues.apache.org/jira/browse/HBASE-19965 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > See > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/] > > java.lang.AssertionError: expected:<2> but was:<3> at > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359) > > Merge regions not work. The table still have 3 regions after the > MergeRegionsProcedure finished. > The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because > the MergeRegionsProcedure pid=138 start work first, so the balance need wait > for the lock. But after merge regions finished, the MoveRegionProcedure > pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new > region server. This is not right. The MoveRegionProcedure should skip to > assign a region which was marked as offline. Or we should clear the merged > regions' procedure when MergeRegionsProcedure finished. > > Logs: > 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1] > master.HMaster(1454): balance > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 > 2018-02-08 16:24:44,608 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885] > procedure2.ProcedureExecutor(868): Stored pid=138, > state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure > table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4, > 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false > .. > 2018-02-08 16:24:50,111 INFO [PEWorker-13] > procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS; > MergeTableRegionsProcedure table=testMergeRegions, > regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479], > forcibly=false in 5.5710sec > 2018-02-08 16:24:50,113 INFO [PEWorker-13] > procedure.MasterProcedureScheduler(813): pid=139, > state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 testMergeRegions > testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[ https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19965: -- Priority: Critical (was: Major) > Fix flaky TestAsyncRegionAdminApi > - > > Key: HBASE-19965 > URL: https://issues.apache.org/jira/browse/HBASE-19965 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-2 > > > See > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/] > > java.lang.AssertionError: expected:<2> but was:<3> at > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359) > > Merge regions not work. The table still have 3 regions after the > MergeRegionsProcedure finished. > The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because > the MergeRegionsProcedure pid=138 start work first, so the balance need wait > for the lock. But after merge regions finished, the MoveRegionProcedure > pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new > region server. This is not right. The MoveRegionProcedure should skip to > assign a region which was marked as offline. Or we should clear the merged > regions' procedure when MergeRegionsProcedure finished. > > Logs: > 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1] > master.HMaster(1454): balance > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 > 2018-02-08 16:24:44,608 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885] > procedure2.ProcedureExecutor(868): Stored pid=138, > state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure > table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4, > 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false > .. > 2018-02-08 16:24:50,111 INFO [PEWorker-13] > procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS; > MergeTableRegionsProcedure table=testMergeRegions, > regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479], > forcibly=false in 5.5710sec > 2018-02-08 16:24:50,113 INFO [PEWorker-13] > procedure.MasterProcedureScheduler(813): pid=139, > state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 testMergeRegions > testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[ https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19965: -- Fix Version/s: 2.0.0-beta-2 > Fix flaky TestAsyncRegionAdminApi > - > > Key: HBASE-19965 > URL: https://issues.apache.org/jira/browse/HBASE-19965 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > See > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/] > > java.lang.AssertionError: expected:<2> but was:<3> at > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359) > > Merge regions not work. The table still have 3 regions after the > MergeRegionsProcedure finished. > The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because > the MergeRegionsProcedure pid=138 start work first, so the balance need wait > for the lock. But after merge regions finished, the MoveRegionProcedure > pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new > region server. This is not right. The MoveRegionProcedure should skip to > assign a region which was marked as offline. Or we should clear the merged > regions' procedure when MergeRegionsProcedure finished. > > Logs: > 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1] > master.HMaster(1454): balance > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 > 2018-02-08 16:24:44,608 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885] > procedure2.ProcedureExecutor(868): Stored pid=138, > state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure > table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4, > 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false > .. > 2018-02-08 16:24:50,111 INFO [PEWorker-13] > procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS; > MergeTableRegionsProcedure table=testMergeRegions, > regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479], > forcibly=false in 5.5710sec > 2018-02-08 16:24:50,113 INFO [PEWorker-13] > procedure.MasterProcedureScheduler(813): pid=139, > state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 testMergeRegions > testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-19876:
---
Attachment: HBASE-19876.branch-1.2.v0.patch
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-19876:
---
Status: Patch Available (was: Open)
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19735) Create a minimal "client" tarball installation
[ https://issues.apache.org/jira/browse/HBASE-19735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358669#comment-16358669 ] Josh Elser commented on HBASE-19735: Looks like org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded is what failed (which I can't see how it would be related to this change). [~mdrob], [~stack]: thing this is good for master? > Create a minimal "client" tarball installation > -- > > Key: HBASE-19735 > URL: https://issues.apache.org/jira/browse/HBASE-19735 > Project: HBase > Issue Type: New Feature >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-19735.000.patch, HBASE-19735.001.branch-2.patch, > HBASE-19735.002.branch-2.patch, HBASE-19735.003.patch, HBASE-19735.004.patch > > > We're moving ourselves towards more controlled dependencies. A logical next > step is to try to do the same for our "binary" artifacts that we create > during releases. > There is code (our's and our dependency's) which the HMaster and RegionServer > require which, obviously, clients do not need. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358708#comment-16358708 ] stack commented on HBASE-19966: --- Copy to a local variable just in case it gets set to null between check and use? mvcc.complete(walKey.getWriteEntry()); Can fix on commit. +1 (Is failed test related?) > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19923) Reset peer state and config when refresh replication source failed
[ https://issues.apache.org/jira/browse/HBASE-19923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358722#comment-16358722 ] Hudson commented on HBASE-19923: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4555 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4555/]) HBASE-19923 Reset peer state and config when refresh replication source (zghao: rev 2b63af376edbca53eb0475e929478bd38b580aae) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/PeerProcedureHandlerImpl.java * (edit) hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerImpl.java > Reset peer state and config when refresh replication source failed > -- > > Key: HBASE-19923 > URL: https://issues.apache.org/jira/browse/HBASE-19923 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-19923.master.001.patch, > HBASE-19923.master.001.patch, HBASE-19923.master.002.patch, > HBASE-19923.master.003.patch > > > Now we use procedure for replication. When peer state changed, the RS will > read peer state from storage to cache. If RS found the peer state changed, > then it will refresh replication source. If refresh failed, the Master will > retry the procedure. Then RS will read peer state again, but now the peer > state in cache is right. So it don't refresh replication source.. So we > need reset the peer state to old peer state when refresh failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358721#comment-16358721 ] Hudson commented on HBASE-19964: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4555 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4555/]) HBASE-19964 TestWriteHeavyIncrementObserver fails (zhangduo: rev 05279d2f5f45f170267c85e16e8472773094a59f) * (edit) hbase-examples/src/test/java/org/apache/hadoop/hbase/coprocessor/example/TestWriteHeavyIncrementObserver.java > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358731#comment-16358731 ] huaxiang sun commented on HBASE-15381: -- Thanks [~jingcheng...@intel.com], I am taking over. > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du >Priority: Major > Attachments: HBASE-15381-v2.patch, HBASE-15381-v3.patch, > HBASE-15381-v4.patch, HBASE-15381-v5.patch, HBASE-15381-v6.patch, > HBASE-15381.patch, mob distributed compaction design-v2.pdf, mob distributed > compaction design.pdf > > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun reassigned HBASE-15381: Assignee: huaxiang sun (was: Jingcheng Du) > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: huaxiang sun >Priority: Major > Attachments: HBASE-15381-v2.patch, HBASE-15381-v3.patch, > HBASE-15381-v4.patch, HBASE-15381-v5.patch, HBASE-15381-v6.patch, > HBASE-15381.patch, mob distributed compaction design-v2.pdf, mob distributed > compaction design.pdf > > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19964) TestWriteHeavyIncrementObserver fails
[ https://issues.apache.org/jira/browse/HBASE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19964: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 Resolving. It dropped off the flakies list. Thanks [~Apache9] > TestWriteHeavyIncrementObserver fails > - > > Key: HBASE-19964 > URL: https://issues.apache.org/jira/browse/HBASE-19964 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19964.patch > > > https://builds.apache.org/job/HBASE-Flaky-Tests/25704/artifact/hbase-examples/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.example.TestWriteHeavyIncrementObserver-output.txt/*view*/ > I could reproduce it locally. Let me dig more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358803#comment-16358803
]
Josh Elser commented on HBASE-19863:
[~ram_krish], [~anoop.hbase], seems to me that you guys understand the problem
much better than I do. Any chance I could ask you to keep up on the reviews? I
am worried that I don't yet understand the problem well enough to give a proper
review alone.
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: HBASE-19863-branch1.patch, HBASE-19863-test.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The query matcher will then just skip this fake
> // key/value and the store scanner will progress to the next column. This
> // is obviously not a "real real" seek, but unlike the fake KV earlier in
> // this method, we want this to be propagated to ScanQueryMatcher.
> {noformat}
>
> For store B we would set it to fake 10:0/C3/createFirstOnRowColTS()/Maximum
> to skip C3 entirely.
> After that we start searching for qualifier C5 using seekOrSkipToNextColumn
> which run first trySkipToNextColumn:
> {noformat}
> protected boolean trySkipToNe
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358840#comment-16358840
]
Hadoop QA commented on HBASE-19876:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
15s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} hbase-server: The patch generated 0 new + 135
unchanged - 1 fixed = 135 total (was 136) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
27s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 12s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
53s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 86m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {colo
[jira] [Updated] (HBASE-19967) Add Major Compaction Tool options for off-peak / on-peak hours
[ https://issues.apache.org/jira/browse/HBASE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19967: --- Description: After HBASE-19528 an operator can disable automatic major compaction and more intelligently manage major compaction impact on cluster operations with an external tool that drives the compaction activity. This tool can be invoked at whatever interval is desirable, and can restrict activity by table and column family, with given concurrency of regionservers compacting at a given time. Add Major Compaction Tool options for off-peak / on-peak hours. Allow for definition of "off-peak" as a time range bounded by two points on a 24 hour clock, and for two concurrency target for global compaction activity, one for the off-peak interval, the other for the remainder. was: After HBASE-19528 an operator can disable automatic major compaction and more intelligently manage major compaction impact on cluster operations with an external tool that drives the compaction activity. This tool can be invoked at whatever schedule is desirable, and can restrict activity by table and column family, with given concurrency of regionservers compacting at a given time. Add Major Compaction Tool options for off-peak / on-peak hours. Allow for definition of "off-peak" as a time range bounded by two points on a 24 hour clock, and for two concurrency target for global compaction activity, one for the off-peak interval, the other for the remainder. > Add Major Compaction Tool options for off-peak / on-peak hours > -- > > Key: HBASE-19967 > URL: https://issues.apache.org/jira/browse/HBASE-19967 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Priority: Major > > After HBASE-19528 an operator can disable automatic major compaction and more > intelligently manage major compaction impact on cluster operations with an > external tool that drives the compaction activity. This tool can be invoked > at whatever interval is desirable, and can restrict activity by table and > column family, with given concurrency of regionservers compacting at a given > time. > Add Major Compaction Tool options for off-peak / on-peak hours. Allow for > definition of "off-peak" as a time range bounded by two points on a 24 hour > clock, and for two concurrency target for global compaction activity, one for > the off-peak interval, the other for the remainder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19967) Add Major Compaction Tool options for off-peak / on-peak hours
Andrew Purtell created HBASE-19967: -- Summary: Add Major Compaction Tool options for off-peak / on-peak hours Key: HBASE-19967 URL: https://issues.apache.org/jira/browse/HBASE-19967 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell After HBASE-19528 an operator can disable automatic major compaction and more intelligently manage major compaction impact on cluster operations with an external tool that drives the compaction activity. This tool can be invoked at whatever schedule is desirable, and can restrict activity by table and column family, with given concurrency of regionservers compacting at a given time. Add Major Compaction Tool options for off-peak / on-peak hours. Allow for definition of "off-peak" as a time range bounded by two points on a 24 hour clock, and for two concurrency target for global compaction activity, one for the off-peak interval, the other for the remainder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17825) Backup: further optimizations
[
https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359031#comment-16359031
]
Vladimir Rodionov commented on HBASE-17825:
---
{quote}
When false is returned, some files would be deleted while some are left. Is
this desirable?
{quote}
This needs to be added to FT Jira. We need to restore not only meta in case of
failure but physical data as well.
> Backup: further optimizations
> -
>
> Key: HBASE-17825
> URL: https://issues.apache.org/jira/browse/HBASE-17825
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 3.0.0
>
> Attachments: HBASE-17825-v1.patch
>
>
> Some phases of backup and restore can be optimized:
> # WALPlayer support for multiple tables
> # Run DistCp once per all tables during backup/restore
> The eventual goal:
> # 2 M/R jobs per backup/restore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19968) MapReduce test fails with NoClassDefFoundError against hadoop3
Ted Yu created HBASE-19968:
--
Summary: MapReduce test fails with NoClassDefFoundError against
hadoop3
Key: HBASE-19968
URL: https://issues.apache.org/jira/browse/HBASE-19968
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
When running mapreduce tests against hadoop3, I observed the following:
{code}
[ERROR]
testWithMockedMapReduceSingleRegion(org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat)
Time elapsed: 0.024 s <<< ERROR!
java.lang.NoClassDefFoundError:
org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo
at
org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo
at
org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
{code}
This was due to lack of dependency on hadoop-hdfs module in the hadoop-3.0
profile of hbase-mapreduce/pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19968) MapReduce test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-19968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359043#comment-16359043
]
Ted Yu commented on HBASE-19968:
TestTableSnapshotInputFormat passed with the patch against hadoop3.
Running mapreduce tests against hadoop3 now.
> MapReduce test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-19968
> URL: https://issues.apache.org/jira/browse/HBASE-19968
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19968.v1.txt
>
>
> When running mapreduce tests against hadoop3, I observed the following:
> {code}
> [ERROR]
> testWithMockedMapReduceSingleRegion(org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat)
> Time elapsed: 0.024 s <<< ERROR!
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo
> at
> org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo
> at
> org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
> {code}
> This was due to lack of dependency on hadoop-hdfs module in the hadoop-3.0
> profile of hbase-mapreduce/pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19968) MapReduce test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-19968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19968:
---
Attachment: 19968.v1.txt
> MapReduce test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-19968
> URL: https://issues.apache.org/jira/browse/HBASE-19968
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19968.v1.txt
>
>
> When running mapreduce tests against hadoop3, I observed the following:
> {code}
> [ERROR]
> testWithMockedMapReduceSingleRegion(org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat)
> Time elapsed: 0.024 s <<< ERROR!
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo
> at
> org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo
> at
> org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
> {code}
> This was due to lack of dependency on hadoop-hdfs module in the hadoop-3.0
> profile of hbase-mapreduce/pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19968) MapReduce test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-19968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19968:
---
Status: Patch Available (was: Open)
> MapReduce test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-19968
> URL: https://issues.apache.org/jira/browse/HBASE-19968
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 19968.v1.txt
>
>
> When running mapreduce tests against hadoop3, I observed the following:
> {code}
> [ERROR]
> testWithMockedMapReduceSingleRegion(org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat)
> Time elapsed: 0.024 s <<< ERROR!
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo
> at
> org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo
> at
> org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat.testWithMockedMapReduce(TestTableSnapshotInputFormat.java:178)
> {code}
> This was due to lack of dependency on hadoop-hdfs module in the hadoop-3.0
> profile of hbase-mapreduce/pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19949) TestRSGroupsWithACL fails with ExceptionInInitializerError
[
https://issues.apache.org/jira/browse/HBASE-19949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19949:
---
Fix Version/s: 1.4.2
1.5.0
> TestRSGroupsWithACL fails with ExceptionInInitializerError
> --
>
> Key: HBASE-19949
> URL: https://issues.apache.org/jira/browse/HBASE-19949
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: 19949.v1.txt
>
>
> {code}
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL Time elapsed: 0.168 sec
> <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL.(TestRSGroupsWithACL.java:60)
> {code}
> The cause is that HBaseClassTestRule#getTimeoutInSeconds() cannot retrieve
> annotation for test size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359069#comment-16359069
]
stack commented on HBASE-16060:
---
With patch in place...
hbase(main):002:0> is_enabled 'TestTable'
true
0 row(s) in 0.1240 seconds
hbase(main):003:0> is_enabled 'disable'
false
0 row(s) in 0.0230 seconds
hbase(main):004:0> is_disabled 'disable'
true
0 row(s) in 0.0190 seconds
hbase(main):005:0> is_disabled 'TestTable'
false
0 row(s) in 0.0250 seconds
'disable' is a disabled table.
Playing more. The migration of table state from hbase-1.x. zookeepers is
something we want and we have something courtesy of HBASE-13032 but it deleted
all zk table state when done which messes up what this patch is trying to do.
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19949) TestRSGroupsWithACL fails with ExceptionInInitializerError
[
https://issues.apache.org/jira/browse/HBASE-19949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19949:
---
Fix Version/s: (was: 1.4.2)
(was: 1.5.0)
> TestRSGroupsWithACL fails with ExceptionInInitializerError
> --
>
> Key: HBASE-19949
> URL: https://issues.apache.org/jira/browse/HBASE-19949
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: 19949.v1.txt
>
>
> {code}
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL Time elapsed: 0.168 sec
> <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL.(TestRSGroupsWithACL.java:60)
> {code}
> The cause is that HBaseClassTestRule#getTimeoutInSeconds() cannot retrieve
> annotation for test size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19969) Improve FT in merge operation
Vladimir Rodionov created HBASE-19969: - Summary: Improve FT in merge operation Key: HBASE-19969 URL: https://issues.apache.org/jira/browse/HBASE-19969 Project: HBase Issue Type: Sub-task Reporter: Vladimir Rodionov -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov reassigned HBASE-19969: - Assignee: Vladimir Rodionov > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19859) Update download page header for 1.1 EOL
[ https://issues.apache.org/jira/browse/HBASE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359074#comment-16359074 ] Andrew Purtell commented on HBASE-19859: After we move the stable pointer to 1.3.2 I'd like to work towards moving again to 1.4. I'll start a discussion on dev@ at that time to establish acceptance criteria. > Update download page header for 1.1 EOL > --- > > Key: HBASE-19859 > URL: https://issues.apache.org/jira/browse/HBASE-19859 > Project: HBase > Issue Type: Task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19583.patch > > > See example mirror: http://mirrors.ocf.berkeley.edu/apache/hbase/ > They still claim that 1.1 is under active development. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19969) Improve FT in merge operation
[ https://issues.apache.org/jira/browse/HBASE-19969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-19969: -- Description: Some file system operations are not fault tolerant during merge. We delete backup data in a backup file system, then copy new data over to backup destination. Deletes can be partial, copy can fail as well > Improve FT in merge operation > - > > Key: HBASE-19969 > URL: https://issues.apache.org/jira/browse/HBASE-19969 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > Some file system operations are not fault tolerant during merge. We delete > backup data in a backup file system, then copy new data over to backup > destination. Deletes can be partial, copy can fail as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17825) Backup: further optimizations
[
https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359031#comment-16359031
]
Vladimir Rodionov edited comment on HBASE-17825 at 2/9/18 10:59 PM:
{quote}
When false is returned, some files would be deleted while some are left. Is
this desirable?
{quote}
This needs to be added to FT Jira. We need to restore not only meta in case of
failure but physical data as well.
That is the separate issue: HBASE-19969
was (Author: vrodionov):
{quote}
When false is returned, some files would be deleted while some are left. Is
this desirable?
{quote}
This needs to be added to FT Jira. We need to restore not only meta in case of
failure but physical data as well.
> Backup: further optimizations
> -
>
> Key: HBASE-17825
> URL: https://issues.apache.org/jira/browse/HBASE-17825
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 3.0.0
>
> Attachments: HBASE-17825-v1.patch
>
>
> Some phases of backup and restore can be optimized:
> # WALPlayer support for multiple tables
> # Run DistCp once per all tables during backup/restore
> The eventual goal:
> # 2 M/R jobs per backup/restore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19937) Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer are initialized
[
https://issues.apache.org/jira/browse/HBASE-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19937:
---
Fix Version/s: 1.4.2
1.5.0
> Ensure createRSGroupTable be called after ProcedureExecutor and LoadBalancer
> are initialized
> -
>
> Key: HBASE-19937
> URL: https://issues.apache.org/jira/browse/HBASE-19937
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.0.0-beta-2
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19937.branch-2.001.patch,
> HBASE-19937.branch-2.002.patch, HBASE-19937.branch-2.003.patch,
> HBASE-19937.branch-2.004.patch, HBASE-19937.branch-2.005.patch,
> HBASE-19937.branch-2.006.patch, import-order.png
>
>
> hbase:rsgroup table will be created by calling createRSGroupTable when master
> load system coprocessors in
>
> {code:java}
> 844 this.cpHost = new MasterCoprocessorHost(this, this.conf);{code}
> when ProcedureExecutor hasn't been initialized before createRSGroupTable, it
> will encounter Exception as follows,
>
> {noformat}
> Exception in thread
> "org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-localhost,49715,1518088607130"
> java.lang.IllegalArgumentException
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:120)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:847)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at org.apache.hadoop.hbase.master.HMaster.createSystemTable(HMaster.java:1795)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:858)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.waitForGroupTableOnline(RSGroupInfoManagerImpl.java:823)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.run(RSGroupInfoManagerImpl.java:743){noformat}
> while ProcedureExecutor initialized by calling
> {code:java}
> 848 startServiceThreads();{code}
> And LoadBalancer is initialized by calling
> {code:java}
> 868 this.balancer.initialize();{code}
> When LoadBalancer hasn't been initialized before createRSGroupTable, it will
> encounters Exception as follows,
> 2018-02-02,16:12:45,688 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=7, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS;
> CreateTableProcedure table=hbase:rsgroup
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.generateGroupMaps(RSGroupBasedLoadBalancer.java:254)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.roundRobinAssignment(RSGroupBasedLoadBalancer.java:162)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.createRoundRobinAssignProcedures(AssignmentManager.java:603)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:108)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:51)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:182)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1227)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1738)
>
> As a result of CreateTableProcedure.rollbackState, it may then print logs
> warning TableExistsException as follows,
> 2018-02-02,16:12:55,503 WARN
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> Failed to perform check
> java.io.IOException: Failed to create group table.
> org.apache.hadoop.hbase.TableExistsException: hbase:rsgroup
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker.createRSGroupTable(RSGroupInfoManagerImpl.java:877)
>
> After some auto-retries, it loops running the thread RSGroupStartupWorker,
> will print logs as follows,
> 2018-02-02,16:23:17,626 INFO
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker:
> RSGroup table=hbase:rsgroup isOnline=true, regionCount=0, assignCount=0,
[jira] [Updated] (HBASE-17825) Backup: further optimizations
[ https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-17825: -- Attachment: HBASE-17825-v2.patch > Backup: further optimizations > - > > Key: HBASE-17825 > URL: https://issues.apache.org/jira/browse/HBASE-17825 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 3.0.0 > > Attachments: HBASE-17825-v1.patch, HBASE-17825-v2.patch > > > Some phases of backup and restore can be optimized: > # WALPlayer support for multiple tables > # Run DistCp once per all tables during backup/restore > The eventual goal: > # 2 M/R jobs per backup/restore -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17825) Backup: further optimizations
[ https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359076#comment-16359076 ] Vladimir Rodionov commented on HBASE-17825: --- Patch v2 contains minor fixes. Created HBASE-19969 to improve FT during Merge > Backup: further optimizations > - > > Key: HBASE-17825 > URL: https://issues.apache.org/jira/browse/HBASE-17825 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 3.0.0 > > Attachments: HBASE-17825-v1.patch, HBASE-17825-v2.patch > > > Some phases of backup and restore can be optimized: > # WALPlayer support for multiple tables > # Run DistCp once per all tables during backup/restore > The eventual goal: > # 2 M/R jobs per backup/restore -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19968) MapReduce test fails with NoClassDefFoundError against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-19968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359079#comment-16359079
]
Hadoop QA commented on HBASE-19968:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 9m
28s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
7s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19968 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909990/19968.v1.txt |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux babcae924bfd 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 2b63af376e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11469/testReport/ |
| Max. process+thread count | 4179 (vs. ulimit of 1) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11469/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> MapReduce test fails with NoClassDefFoundError against hadoop3
> --
>
> Key: HBASE-19968
> URL: https://issues.apa
[jira] [Updated] (HBASE-19949) TestRSGroupsWithACL fails with ExceptionInInitializerError
[
https://issues.apache.org/jira/browse/HBASE-19949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19949:
---
Fix Version/s: 1.4.2
1.5.0
> TestRSGroupsWithACL fails with ExceptionInInitializerError
> --
>
> Key: HBASE-19949
> URL: https://issues.apache.org/jira/browse/HBASE-19949
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 2.0.0-beta-2, 1.4.2
>
> Attachments: 19949.v1.txt
>
>
> {code}
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL Time elapsed: 0.168 sec
> <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL.(TestRSGroupsWithACL.java:60)
> {code}
> The cause is that HBaseClassTestRule#getTimeoutInSeconds() cannot retrieve
> annotation for test size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359099#comment-16359099
]
stack commented on HBASE-16060:
---
Ok. First set up cluster with hbase-1 data. Shut it down. Start hbase-2 over
hbase-1 data. In logs I see these messages:
2018-02-09 15:28:40,980 INFO [master/ve0524:16000]
master.MirroringTableStateManager: Migrating table state from zookeeper to
hbase:meta; tableName=enabled2, state=ENABLED
2018-02-09 15:28:41,166 INFO [master/ve0524:16000] hbase.MetaTableAccessor:
Updated table enabled2 state to ENABLED in META
2018-02-09 15:28:41,177 INFO [master/ve0524:16000]
master.MirroringTableStateManager: Migrating table state from zookeeper to
hbase:meta; tableName=hbase:namespace, state=ENABLED
2018-02-09 15:28:41,189 INFO [master/ve0524:16000] hbase.MetaTableAccessor:
Updated table hbase:namespace state to ENABLED in META
2018-02-09 15:28:41,200 INFO [master/ve0524:16000]
master.MirroringTableStateManager: Purged table state entry for table not in
hbase:meta: disable
2018-02-09 15:28:41,210 INFO [master/ve0524:16000]
master.MirroringTableStateManager: Migrating table state from zookeeper to
hbase:meta; tableName=disabled, state=DISABLED
...
so migration of table state from zk to hbase:meta seems to work. Restart and no
messages (though we are doing the work to see if we need to migrate states... ).
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359106#comment-16359106
]
stack commented on HBASE-16060:
---
Hmm.. With the patch I can now create tables using a 1.x client. I can get
detailed status too. Seems like most function is now working (I've not done
comprehensive test).
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16060:
--
Attachment: HBASE-16060.branch-2.002.patch
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch, HBASE-16060.branch-2.002.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16060:
--
Release Note:
By default, we mirror table state to zookeeper so hbase-1.x clients will work
against an hbase-2 cluster (With this patch, hbase-1.x clients can do most
Admin functions including table create; hbase-1.x clients can do all Table/DML
against hbase-2 cluster).
Flag to disable mirroring is hbase.mirror.table.state.to.zookeeper; set it to
false in Configuration.
Related, Master on startup will look to see if there are table state znodes
left over by an hbase-1 instance. If any found, it will migrate the table state
to hbase-2 setting the state into the hbase:meta table where table state is now
kept. We will do this check on every Master start. Notion is that this will be
overall beneficial with low impediment. To disable the migration check, set
hbase.migrate.table.state.from.zookeeper to false.
Status: Patch Available (was: Open)
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch, HBASE-16060.branch-2.002.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-19172) [rolling upgrade] Migrate region state from zk to hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19172. --- Resolution: Duplicate HBASE-13032 did a form of this years ago. HBASE-13032 was refactored in HBASE-16060 which adds a unit test to prove this works. It also includes notes on how we verified this working on a cluster deploy. Resolving as duplicate. > [rolling upgrade] Migrate region state from zk to hbase:meta > > > Key: HBASE-19172 > URL: https://issues.apache.org/jira/browse/HBASE-19172 > Project: HBase > Issue Type: Sub-task > Components: migration >Reporter: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > > HBASE-19165 is about starting a branch-2 over branch-1 dataset. The Master > when it comes up, goes to read the 'state' of the region from hbase:meta > because this is how region state is kept now by AMv2 only, there is no state > in hbase:meta on a fresh start. In hbase1, region state was kept in master > memory persisted to zk (rather than hbase:meta) while the region was in > transition. > We need to add to master a self-migration facility. If a region has no state > in hbase:meta, then look in zk to see if it RIT. If nothing in RIT, ask the > pertinent regionserver if it has the region. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359123#comment-16359123
]
stack commented on HBASE-16060:
---
.001 does the below. May I have a review please?
HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.
The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.
D
hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
Unused.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.
A
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java
M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.
M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.
A
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
Test migration and mirroring.
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch, HBASE-16060.branch-2.002.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19942) Fix flaky TestSimpleRpcScheduler
[ https://issues.apache.org/jira/browse/HBASE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19942: -- Resolution: Fixed Status: Resolved (was: Patch Available) Re-resolving after [~zghaobac] 's push. This test no longer shows on the flakies list. Thank you [~zghaobac] > Fix flaky TestSimpleRpcScheduler > > > Key: HBASE-19942 > URL: https://issues.apache.org/jira/browse/HBASE-19942 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19942.branch-2.001.patch, > HBASE-19942.branch-2.addendum.patch, HBASE-19942.master.001.patch, > HBASE-19942.master.addendum.patch > > > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/1387/testReport/junit/org.apache.hadoop.hbase.ipc/TestSimpleRpcScheduler/testSoftAndHardQueueLimits/ > > h3. Stacktrace > java.lang.AssertionError at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:451) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19966: -- Attachment: 0001-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19966: -- Attachment: (was: 0001-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch) > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19966: -- Attachment: HBASE-19966.patch > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch, HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359132#comment-16359132 ] stack commented on HBASE-19966: --- Ignore above comment. Not going to happen. Meantime, retrying while Duo having his breakfast. > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch, HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359205#comment-16359205
]
Hadoop QA commented on HBASE-16060:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} hbase-client: The patch generated 0 new + 26
unchanged - 1 fixed = 26 total (was 27) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} hbase-server: The patch generated 16 new + 294
unchanged - 5 fixed = 310 total (was 299) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
53s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 41s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[
https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359206#comment-16359206
]
Hadoop QA commented on HBASE-19966:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
56s{color} | {color:red} hbase-server: The patch generated 3 new + 13 unchanged
- 6 fixed = 16 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
15s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
13s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
23s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 98m
55s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19966 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910005/HBASE-19966.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 05e8a8f1b0fa 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 2b63af376e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11471/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/114
[jira] [Created] (HBASE-19970) Remove unused functions from TableAuthManager
Appy created HBASE-19970: Summary: Remove unused functions from TableAuthManager Key: HBASE-19970 URL: https://issues.apache.org/jira/browse/HBASE-19970 Project: HBase Issue Type: Task Reporter: Appy Assignee: Appy Functions deleted in TableAuthManager: - setTableUserPermissions - setTableGroupPermissions - setNamespaceUserPermissions - setNamespaceGroupPermissions - writeTableToZooKeeper - writeNamespaceToZooKeeper To make sure it was not a bug, and that relevant functionality moved to some alternate code path, tried to find out why and when these functions went out of use. But just couldn't figure out...until i reached the patch which added them. Looks like they were dead functions to start with :) Jira which added them: HBASE-8409. Commit id: ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19970) Remove unused functions from TableAuthManager
[ https://issues.apache.org/jira/browse/HBASE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359209#comment-16359209 ] Appy commented on HBASE-19970: -- I was trying to understand read/write paths for ACL meta data, looks like the write path is: grant() rpc --> execute CP service on ACL table's region --> write data to ACL region --> postGet hook --> use ZKPermissionsWatcher to write new data to zk > Remove unused functions from TableAuthManager > - > > Key: HBASE-19970 > URL: https://issues.apache.org/jira/browse/HBASE-19970 > Project: HBase > Issue Type: Task >Reporter: Appy >Assignee: Appy >Priority: Minor > > Functions deleted in TableAuthManager: > - setTableUserPermissions > - setTableGroupPermissions > - setNamespaceUserPermissions > - setNamespaceGroupPermissions > - writeTableToZooKeeper > - writeNamespaceToZooKeeper > To make sure it was not a bug, and that relevant functionality moved to some > alternate code path, tried to find out why and when these functions went out > of use. But just couldn't figure out...until i reached the patch which added > them. Looks like they were dead functions to start with :) > Jira which added them: HBASE-8409. Commit id: > ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19970) Remove unused functions from TableAuthManager
[ https://issues.apache.org/jira/browse/HBASE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359209#comment-16359209 ] Appy edited comment on HBASE-19970 at 2/10/18 2:37 AM: --- I was trying to understand read/write paths for ACL meta data, looks like the write path is as following (and doesn't involve deleted functions in any way): grant() rpc --> execute CP service on ACL table's region --> write data to ACL region --> postGet hook --> use ZKPermissionsWatcher to write new data to zk was (Author: appy): I was trying to understand read/write paths for ACL meta data, looks like the write path is: grant() rpc --> execute CP service on ACL table's region --> write data to ACL region --> postGet hook --> use ZKPermissionsWatcher to write new data to zk > Remove unused functions from TableAuthManager > - > > Key: HBASE-19970 > URL: https://issues.apache.org/jira/browse/HBASE-19970 > Project: HBase > Issue Type: Task >Reporter: Appy >Assignee: Appy >Priority: Minor > > Functions deleted in TableAuthManager: > - setTableUserPermissions > - setTableGroupPermissions > - setNamespaceUserPermissions > - setNamespaceGroupPermissions > - writeTableToZooKeeper > - writeNamespaceToZooKeeper > To make sure it was not a bug, and that relevant functionality moved to some > alternate code path, tried to find out why and when these functions went out > of use. But just couldn't figure out...until i reached the patch which added > them. Looks like they were dead functions to start with :) > Jira which added them: HBASE-8409. Commit id: > ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19970) Remove unused functions from TableAuthManager
[ https://issues.apache.org/jira/browse/HBASE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19970: - Attachment: HBASE-19970.master.001.patch > Remove unused functions from TableAuthManager > - > > Key: HBASE-19970 > URL: https://issues.apache.org/jira/browse/HBASE-19970 > Project: HBase > Issue Type: Task >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19970.master.001.patch > > > Functions deleted in TableAuthManager: > - setTableUserPermissions > - setTableGroupPermissions > - setNamespaceUserPermissions > - setNamespaceGroupPermissions > - writeTableToZooKeeper > - writeNamespaceToZooKeeper > To make sure it was not a bug, and that relevant functionality moved to some > alternate code path, tried to find out why and when these functions went out > of use. But just couldn't figure out...until i reached the patch which added > them. Looks like they were dead functions to start with :) > Jira which added them: HBASE-8409. Commit id: > ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19970) Remove unused functions from TableAuthManager
[ https://issues.apache.org/jira/browse/HBASE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359215#comment-16359215 ] Appy commented on HBASE-19970: -- Ping [~stack] > Remove unused functions from TableAuthManager > - > > Key: HBASE-19970 > URL: https://issues.apache.org/jira/browse/HBASE-19970 > Project: HBase > Issue Type: Task >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19970.master.001.patch > > > Functions deleted in TableAuthManager: > - setTableUserPermissions > - setTableGroupPermissions > - setNamespaceUserPermissions > - setNamespaceGroupPermissions > - writeTableToZooKeeper > - writeNamespaceToZooKeeper > To make sure it was not a bug, and that relevant functionality moved to some > alternate code path, tried to find out why and when these functions went out > of use. But just couldn't figure out...until i reached the patch which added > them. Looks like they were dead functions to start with :) > Jira which added them: HBASE-8409. Commit id: > ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19970) Remove unused functions from TableAuthManager
[ https://issues.apache.org/jira/browse/HBASE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19970: - Status: Patch Available (was: Open) > Remove unused functions from TableAuthManager > - > > Key: HBASE-19970 > URL: https://issues.apache.org/jira/browse/HBASE-19970 > Project: HBase > Issue Type: Task >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19970.master.001.patch > > > Functions deleted in TableAuthManager: > - setTableUserPermissions > - setTableGroupPermissions > - setNamespaceUserPermissions > - setNamespaceGroupPermissions > - writeTableToZooKeeper > - writeNamespaceToZooKeeper > To make sure it was not a bug, and that relevant functionality moved to some > alternate code path, tried to find out why and when these functions went out > of use. But just couldn't figure out...until i reached the patch which added > them. Looks like they were dead functions to start with :) > Jira which added them: HBASE-8409. Commit id: > ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359218#comment-16359218 ] stack commented on HBASE-19966: --- Failure is in hbase-thrift schema download [ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) on project hbase-thrift: Failed to install metadata org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got / (position: END_TAG seen ...\n/... @25:2) -> [Help 1] Let me push. > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch, HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19966: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2 and master. Thanks [~Apache9] > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch, HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19950) Introduce a ColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359224#comment-16359224
]
Chia-Ping Tsai commented on HBASE-19950:
1) Why we don't allow user to pass the null qualifier? The null qualifier is
valid in hbase. Perhaps we can convert the passed null qualifier to empty array?
{code:java}
+ public ColumnValueFilter(final byte[] family, final byte[] qualifier,
+ final CompareOperator op,
+ final ByteArrayComparable comparator) {
+ this.family = Preconditions.checkNotNull(family, "family should not be
null.");
+ this.qualifier = Preconditions.checkNotNull(qualifier, "qualifier should not
be null");
+ this.op = Preconditions.checkNotNull(op, "CompareOperator should not be
null");
+ this.comparator = Preconditions.checkNotNull(comparator, "Comparator should
not be null");
+ }{code}
2) ColumnValueFilter doesn't filter the cell by the row, right?
{code:java}
+ @Override
+ public boolean hasFilterRow() {
+ return true;
+ }{code}
3) Seems to me all fields can be final?
{code:java}
+ private byte[] family;
+ private byte[] qualifier;
+ private CompareOperator op;
+ private ByteArrayComparable comparator;{code}
4) Please add the check of NO_OP. Otherwise, the compare will throw the
RuntimeException
{code:java}
+ private boolean compareValue(final CompareOperator op, final
ByteArrayComparable comparator,
+ final Cell cell) {
+ int compareResult = PrivateCellUtil.compareValue(cell, comparator);
+ return CompareFilter.compare(op, compareResult);
+ }{code}
> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch,
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when
> specified condition is matched, this new filter will return the matched cell
> only. There're already some discussions in HBASE-19824.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359225#comment-16359225
]
Chia-Ping Tsai commented on HBASE-19876:
The failed tests pass locally...Try the patch for master. Ping for reviews also.
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-19876:
---
Attachment: HBASE-19876.v6.patch
> The exception happening in converting pb mutation to hbase.mutation messes up
> the CellScanner
> -
>
> Key: HBASE-19876
> URL: https://issues.apache.org/jira/browse/HBASE-19876
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
>
> Attachments: HBASE-19876.branch-1.2.v0.patch,
> HBASE-19876.master.001.patch, HBASE-19876.v0.patch, HBASE-19876.v1.patch,
> HBASE-19876.v2.patch, HBASE-19876.v3.patch, HBASE-19876.v3.patch,
> HBASE-19876.v3.patch, HBASE-19876.v3.patch, HBASE-19876.v4.patch,
> HBASE-19876.v5.patch, HBASE-19876.v6.patch
>
>
> {code:java}
> 2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
> client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
> attempt=6/16 failed=20ops, last
> exception=org.apache.hadoop.hbase.client.WrongRowIOException:
> org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
> match the original one aaa
> at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
> at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
> I noticed this bug when testing the table space quota.
> When rs are converting pb mutation to hbase.mutation, the quota exception or
> cell exception may be thrown.
> {code}
> Unable to find source-code formatter for language:
> rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
> applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
> java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
> rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
> (ClientProtos.Action action: mutations) {
> MutationProto m = action.getMutation();
> Mutation mutation;
> if (m.getMutateType() == MutationType.PUT) {
> mutation = ProtobufUtil.toPut(m, cells);
> batchContainsPuts = true;
> } else {
> mutation = ProtobufUtil.toDelete(m, cells);
> batchContainsDelete = true;
> }
> mutationActionMap.put(mutation, action);
> mArray[i++] = mutation;
> checkCellSizeLimit(region, mutation);
> // Check if a space quota disallows this mutation
> spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
> quota.addMutation(mutation);
> }
> {code}
> rs has caught the exception but it doesn't have the cellscanner skip the
> failed cells.
> {code:java}
> } catch (IOException ie) {
> if (atomic) {
> throw ie;
> }
> for (Action mutation : mutations) {
> builder.addResultOrException(getResultOrException(ie,
> mutation.getIndex()));
> }
> }
> {code}
> The bug results in the WrongRowIOException to remaining mutations since they
> refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[
https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359226#comment-16359226
]
Guanghao Zhang commented on HBASE-19965:
For the regions belong to disabled table or splitted region, we skip to assign
it in AssignProcedure.startTransition method. Maybe we can use the same pattern
for merged region. Thanks [~stack] for pick up this. :-)
{code:java}
// Don't assign if table is in disabling of disabled state.
TableStateManager tsm = env.getMasterServices().getTableStateManager();
TableName tn = regionNode.getRegionInfo().getTable();
if (tsm.isTableState(tn, TableState.State.DISABLING,
TableState.State.DISABLED)) {
LOG.info("Table " + tn + " state=" + tsm.getTableState(tn) + ", skipping
" + this);
return false;
}
// If the region is SPLIT, we can't assign it. But state might be CLOSED,
rather than
// SPLIT which is what a region gets set to when Unassigned as part of
SPLIT. FIX.
if (regionNode.isInState(State.SPLIT) ||
(regionNode.getRegionInfo().isOffline() &&
regionNode.getRegionInfo().isSplit())) {
LOG.info("SPLIT, cannot be assigned; " + this + "; " + regionNode +
"; hri=" + regionNode.getRegionInfo());
return false;
}
{code}
> Fix flaky TestAsyncRegionAdminApi
> -
>
> Key: HBASE-19965
> URL: https://issues.apache.org/jira/browse/HBASE-19965
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> See
> [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/]
>
> java.lang.AssertionError: expected:<2> but was:<3> at
> org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359)
>
> Merge regions not work. The table still have 3 regions after the
> MergeRegionsProcedure finished.
> The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because
> the MergeRegionsProcedure pid=138 start work first, so the balance need wait
> for the lock. But after merge regions finished, the MoveRegionProcedure
> pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new
> region server. This is not right. The MoveRegionProcedure should skip to
> assign a region which was marked as offline. Or we should clear the merged
> regions' procedure when MergeRegionsProcedure finished.
>
> Logs:
> 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1]
> master.HMaster(1454): balance
> hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.,
> source=cd4730e3eae2,39077,1518106776411,
> destination=cd4730e3eae2,40578,1518106776318
> 2018-02-08 16:24:44,608 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885]
> procedure2.ProcedureExecutor(868): Stored pid=138,
> state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure
> table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4,
> 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false
> ..
> 2018-02-08 16:24:50,111 INFO [PEWorker-13]
> procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS;
> MergeTableRegionsProcedure table=testMergeRegions,
> regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479],
> forcibly=false in 5.5710sec
> 2018-02-08 16:24:50,113 INFO [PEWorker-13]
> procedure.MasterProcedureScheduler(813): pid=139,
> state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.,
> source=cd4730e3eae2,39077,1518106776411,
> destination=cd4730e3eae2,40578,1518106776318 testMergeRegions
> testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19966) The WriteEntry for WALKey maybe null if we failed to call WAL.append
[ https://issues.apache.org/jira/browse/HBASE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359230#comment-16359230 ] Duo Zhang commented on HBASE-19966: --- Thanks for taking care of this sir. > The WriteEntry for WALKey maybe null if we failed to call WAL.append > > > Key: HBASE-19966 > URL: https://issues.apache.org/jira/browse/HBASE-19966 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19966.patch, HBASE-19966.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16060:
--
Attachment: HBASE-16060.branch-2.003.patch
> 1.x clients cannot access table state talking to 2.0 cluster
>
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-2
>
> Attachments:
> 0002-HBASE-16060-1.x-clients-cannot-access-table-state-ta.patch,
> HBASE-16060.branch-2.001.patch, HBASE-16060.branch-2.002.patch,
> HBASE-16060.branch-2.003.patch
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients
> talking to 2.0 cluster cannot access the table state. This causes some weird
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and
> {{Admin.isTableDisabled()}} both return false.
> One option we can do is to add code in 1.x clients so that they can access
> the table state in meta if needed. Otherwise, we can mirror the table state
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so
> that any 1.x client can still work correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19965) Fix flaky TestAsyncRegionAdminApi
[ https://issues.apache.org/jira/browse/HBASE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359234#comment-16359234 ] stack commented on HBASE-19965: --- Thank you [~zghaobac] Let me look tomorrow. I came across another instance today where on restart, if CLOSED, we presume an outstanding assign though the table is disabled This seems similar. Let me study it. Thanks for the research. > Fix flaky TestAsyncRegionAdminApi > - > > Key: HBASE-19965 > URL: https://issues.apache.org/jira/browse/HBASE-19965 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-2 > > > See > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2/284/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncRegionAdminApi/testMergeRegions_0_/] > > java.lang.AssertionError: expected:<2> but was:<3> at > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:359) > > Merge regions not work. The table still have 3 regions after the > MergeRegionsProcedure finished. > The master start balance region 9e2773ba1efba79a2defa276e9a26ed4. But because > the MergeRegionsProcedure pid=138 start work first, so the balance need wait > for the lock. But after merge regions finished, the MoveRegionProcedure > pid=139 start work and assign 9e2773ba1efba79a2defa276e9a26ed4 to a new > region server. This is not right. The MoveRegionProcedure should skip to > assign a region which was marked as offline. Or we should clear the merged > regions' procedure when MergeRegionsProcedure finished. > > Logs: > 2018-02-08 16:24:44,608 INFO [master/cd4730e3eae2:0.Chore.1] > master.HMaster(1454): balance > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 > 2018-02-08 16:24:44,608 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=37885] > procedure2.ProcedureExecutor(868): Stored pid=138, > state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE; MergeTableRegionsProcedure > table=testMergeRegions, regions=[9e2773ba1efba79a2defa276e9a26ed4, > 8f8fd5cd032313e1aadb83e31e1b7479], forcibly=false > .. > 2018-02-08 16:24:50,111 INFO [PEWorker-13] > procedure2.ProcedureExecutor(1249): Finished pid=138, state=SUCCESS; > MergeTableRegionsProcedure table=testMergeRegions, > regions=[9e2773ba1efba79a2defa276e9a26ed4, 8f8fd5cd032313e1aadb83e31e1b7479], > forcibly=false in 5.5710sec > 2018-02-08 16:24:50,113 INFO [PEWorker-13] > procedure.MasterProcedureScheduler(813): pid=139, > state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure > hri=testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4., > source=cd4730e3eae2,39077,1518106776411, > destination=cd4730e3eae2,40578,1518106776318 testMergeRegions > testMergeRegions,,1518107079782.9e2773ba1efba79a2defa276e9a26ed4. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19920) TokenUtil.obtainToken unnecessarily creates a local directory
[
https://issues.apache.org/jira/browse/HBASE-19920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359248#comment-16359248
]
Mike Drob commented on HBASE-19920:
---
Not sure why it couldn't comment...
https://builds.apache.org/job/PreCommit-HBASE-Build/11457 ran against v8
{noformat}
02:35:47.560
02:35:47.560
02:35:47.560 Adding comment to JIRA
02:35:47.560
02:35:47.560
02:35:47.560
02:35:47.560
02:37:11.642
02:37:11.642
02:37:11.642 +1 overall
02:37:11.642
02:37:11.764 | Vote | Subsystem | Runtime | Comment
02:37:11.764
02:37:11.822 | 0 | reexec | 0m 12s | Docker mode activated.
02:37:11.853 | | || Prechecks
02:37:11.883 | 0 | findbugs | 0m 0s | Findbugs executables are
not available.
02:37:11.910 | +1 | hbaseanti | 0m 0s | Patch does not have any
anti-patterns.
02:37:11.939 | +1 |@author | 0m 0s | The patch does not contain
any @author
02:37:11.939 | | || tags.
02:37:11.967 | +1 | test4tests | 0m 0s | The patch appears to
include 1 new or
02:37:11.967 | | || modified test files.
02:37:11.994 | | || master Compile Tests
02:37:12.026 | 0 | mvndep | 0m 23s | Maven dependency ordering
for branch
02:37:12.061 | +1 | mvninstall | 4m 25s | master passed
02:37:12.091 | +1 |compile | 1m 3s | master passed
02:37:12.116 | +1 | checkstyle | 1m 39s | master passed
02:37:12.145 | +1 | shadedjars | 6m 21s | branch has no errors when
building our
02:37:12.145 | | || shaded downstream
artifacts.
02:37:12.173 | +1 |javadoc | 0m 48s | master passed
02:37:12.197 | | || Patch Compile Tests
02:37:12.221 | 0 | mvndep | 0m 14s | Maven dependency ordering
for patch
02:37:12.252 | +1 | mvninstall | 4m 21s | the patch passed
02:37:12.281 | +1 |compile | 1m 2s | the patch passed
02:37:12.307 | +1 | javac | 1m 2s | the patch passed
02:37:12.339 | +1 | checkstyle | 1m 37s | the patch passed
02:37:12.372 | +1 | whitespace | 0m 0s | The patch has no
whitespace issues.
02:37:12.401 | +1 | shadedjars | 4m 42s | patch has no errors when
building our
02:37:12.401 | | || shaded downstream
artifacts.
02:37:12.437 | +1 |hadoopcheck | 18m 58s | Patch does not cause any
errors with
02:37:12.437 | | || Hadoop 2.6.5 2.7.4 or
3.0.0.
02:37:12.480 | +1 |javadoc | 0m 50s | the patch passed
02:37:12.509 | | || Other Tests
02:37:12.538 | +1 | unit | 3m 4s | hbase-client in the patch
passed.
02:37:12.563 | +1 | unit | 111m 12s | hbase-server in the patch
passed.
02:37:12.593 | +1 | asflicense | 0m 40s | The patch does not
generate ASF License
02:37:12.593 | | || warnings.
02:37:12.625 | | | 155m 36s |
02:37:12.626
02:37:12.626
02:37:12.626 || Subsystem || Report/Notes ||
02:37:12.626
02:37:12.632 | Docker | Client=17.05.0-ce Server=17.05.0-ce
Image:yetus/hbase:eee3b01 |
02:37:12.637 | JIRA Issue | HBASE-19920 |
02:37:12.642 | JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12909862/HBASE-19920.v8.patch |
02:37:12.647 | Optional Tests | asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
02:37:12.652 | uname | Linux 3b98d43f13bd 3.13.0-139-generic #188-Ubuntu SMP
Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
02:37:12.656 | Build tool | maven |
02:37:12.660 | Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
02:37:12.663 | git revision | master / f1a81618fd |
02:37:12.688 | maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
02:37:12.688 | Default Java | 1.8.0_151 |
02:37:12.688 | Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11457/testReport/ |
02:37:12.688 | Max. process+thread count | 4404 (vs. ulimit of 1) |
02:37:12.688 | modules | C: hbase-client
[jira] [Commented] (HBASE-19970) Remove unused functions from TableAuthManager
[
https://issues.apache.org/jira/browse/HBASE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359265#comment-16359265
]
Hadoop QA commented on HBASE-19970:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
2s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
31s{color} | {color:red} hbase-server: The patch generated 3 new + 89 unchanged
- 12 fixed = 92 total (was 101) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
10s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 59s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 12s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19970 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910015/HBASE-19970.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 23045f8699f1 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 2b63af376e |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/11472/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11472/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11472/testReport/ |
| Max. process+thread count | 4831 (
[jira] [Commented] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
[
https://issues.apache.org/jira/browse/HBASE-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359276#comment-16359276
]
Hadoop QA commented on HBASE-19876:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
1s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}115m
15s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
51s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19876 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910018/HBASE-19876.v6.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7387a76c9d23 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 32f235abee |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11473/testReport/ |
| Max. process+thread c
[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster
[
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359280#comment-16359280
]
Hadoop QA commented on HBASE-16060:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
51s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
1s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} hbase-client: The patch generated 0 new + 26
unchanged - 1 fixed = 26 total (was 27) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
14s{color} | {color:red} hbase-server: The patch generated 10 new + 294
unchanged - 5 fixed = 304 total (was 299) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 55s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}115m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Updated] (HBASE-19950) Introduce a ColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-19950:
--
Attachment: HBASE-19950.master.004.patch
> Introduce a ColumnValueFilter
> -
>
> Key: HBASE-19950
> URL: https://issues.apache.org/jira/browse/HBASE-19950
> Project: HBase
> Issue Type: Improvement
> Components: Filters
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-19950.master.001.patch,
> HBASE-19950.master.002.patch, HBASE-19950.master.003.patch,
> HBASE-19950.master.004.patch
>
>
> Different from {{SingleColumnValueFilter}} which returns an entire row when
> specified condition is matched, this new filter will return the matched cell
> only. There're already some discussions in HBASE-19824.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
