[jira] [Commented] (HBASE-18658) Purge hokey hbase Service implementation; use (internal) Guava Service instead
[
https://issues.apache.org/jira/browse/HBASE-18658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137967#comment-16137967
]
Hadoop QA commented on HBASE-18658:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
20s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 26s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.locking.TestLockManager |
| | hadoop.hbase.procedure.TestProcedureManager |
| | hadoop.hbase.regionserver.TestHRegionFileSystem |
| | hadoop.hbase.master.balancer.TestRegionLocationFinder |
| | hadoop.hbase.backup.TestBackupHFileCleaner |
| | hadoop.hbase.ipc.TestNettyRpcServer |
| | hadoop.hbase.master.locking.TestLockProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18658 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883203/HBASE-18658.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux aaab8a7431c7 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenk
[jira] [Updated] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
[ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-18375: Attachment: HBASE-18375-V02-branch-2.patch > The pool chunks from ChunkCreator are deallocated while in pool because there > is no reference to them > - > > Key: HBASE-18375 > URL: https://issues.apache.org/jira/browse/HBASE-18375 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-1 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18375-branch-2.patch, HBASE-18375-V01.patch, > HBASE-18375-V02-branch-2.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch, > HBASE-18375-V04.patch, HBASE-18375-V05.patch, HBASE-18375-V06.patch, > HBASE-18375-V07.patch, HBASE-18375-V08.patch, HBASE-18375-V09.patch, > HBASE-18375-V10.patch, HBASE-18375-V11.patch > > > Because MSLAB list of chunks was changed to list of chunk IDs, the chunks > returned back to pool can be deallocated by JVM because there is no reference > to them. The solution is to protect pool chunks from GC by the strong map of > ChunkCreator introduced by HBASE-18010. Will prepare the patch today. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18633) Add more info to understand the source/scenario of large batch requests exceeding threshold
[ https://issues.apache.org/jira/browse/HBASE-18633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137952#comment-16137952 ] Vikas Vishwakarma commented on HBASE-18633: --- patch for review & QA run > Add more info to understand the source/scenario of large batch requests > exceeding threshold > --- > > Key: HBASE-18633 > URL: https://issues.apache.org/jira/browse/HBASE-18633 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Vikas Vishwakarma >Assignee: Vikas Vishwakarma > Attachments: HBASE-18633.master.001.patch > > > In our controlled test env, we are seeing frequent Large batch operation > detected warnings (as implemented in HBASE-18023). > We are not running any client with large batch sizes on this test env, but we > start seeing these warnings after some runtime. Maybe it is caused due to > some error / retry scenario. Could also be related to Phoenix index updates > based on surrounding activity in the logs. Need to add more info like > table/region name and anything else that will enable debugging the source or > the scenario in which these warnings occur. > 2017-08-12 03:40:33,919 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7108 Client: xxx > 2017-08-12 03:40:34,476 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7096 Client: xxx > 2017-08-12 03:40:34,483 WARN [4,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7091 Client: xxx > 2017-08-12 03:40:35,728 WARN [3,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7102 Client: xxx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18633) Add more info to understand the source/scenario of large batch requests exceeding threshold
[ https://issues.apache.org/jira/browse/HBASE-18633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Vishwakarma updated HBASE-18633: -- Fix Version/s: 3.0.0 Affects Version/s: 3.0.0 Status: Patch Available (was: Open) > Add more info to understand the source/scenario of large batch requests > exceeding threshold > --- > > Key: HBASE-18633 > URL: https://issues.apache.org/jira/browse/HBASE-18633 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1, 3.0.0 >Reporter: Vikas Vishwakarma >Assignee: Vikas Vishwakarma > Fix For: 3.0.0 > > Attachments: HBASE-18633.master.001.patch > > > In our controlled test env, we are seeing frequent Large batch operation > detected warnings (as implemented in HBASE-18023). > We are not running any client with large batch sizes on this test env, but we > start seeing these warnings after some runtime. Maybe it is caused due to > some error / retry scenario. Could also be related to Phoenix index updates > based on surrounding activity in the logs. Need to add more info like > table/region name and anything else that will enable debugging the source or > the scenario in which these warnings occur. > 2017-08-12 03:40:33,919 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7108 Client: xxx > 2017-08-12 03:40:34,476 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7096 Client: xxx > 2017-08-12 03:40:34,483 WARN [4,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7091 Client: xxx > 2017-08-12 03:40:35,728 WARN [3,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7102 Client: xxx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16322) Disable filter for raw scan
[
https://issues.apache.org/jira/browse/HBASE-16322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137953#comment-16137953
]
Duo Zhang commented on HBASE-16322:
---
Finally I found that we do have some usage that filter and raw scan are used
together. For example, for canary, we will use raw scan and a
{{FirstKeyOnlyFilter}}, and the filter still works with delete marker. So maybe
just add a flag to the current filter which indicate whether it can work with
delete marker? Then we can use them in raw scan and also in compaction(for CP
hook).
Thanks.
> Disable filter for raw scan
> ---
>
> Key: HBASE-16322
> URL: https://issues.apache.org/jira/browse/HBASE-16322
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>
> As we will pass delete markers to the filter.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18575) [AMv2] Enable and fix TestRestartCluster#testRetainAssignmentOnRestart on master
[
https://issues.apache.org/jira/browse/HBASE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138010#comment-16138010
]
Hadoop QA commented on HBASE-18575:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 42s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
15s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}112m
19s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
58s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18575 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883262/hbase-18575.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 42632322ed40 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / dcd3e9a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138011#comment-16138011 ] Ashish Singhi commented on HBASE-17442: --- Great work.. We are not moving replication related test codes because they are dependent on hbase-server package classes ? > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0-alpha-3 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.branch-2.001.patch, HBASE-17442.branch-2.001.patch, > HBASE-17442.master.001.patch, HBASE-17442.master.002.patch, > HBASE-17442.master.003.patch, HBASE-17442.master.004.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch, HBASE-17442.v2.patch, > HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
[
https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138013#comment-16138013
]
Hadoop QA commented on HBASE-18375:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18375 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18375 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883275/HBASE-18375-V02-branch-2.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8261/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> The pool chunks from ChunkCreator are deallocated while in pool because there
> is no reference to them
> -
>
> Key: HBASE-18375
> URL: https://issues.apache.org/jira/browse/HBASE-18375
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0-alpha-1
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-18375-branch-2.patch, HBASE-18375-V01.patch,
> HBASE-18375-V02-branch-2.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch,
> HBASE-18375-V04.patch, HBASE-18375-V05.patch, HBASE-18375-V06.patch,
> HBASE-18375-V07.patch, HBASE-18375-V08.patch, HBASE-18375-V09.patch,
> HBASE-18375-V10.patch, HBASE-18375-V11.patch
>
>
> Because MSLAB list of chunks was changed to list of chunk IDs, the chunks
> returned back to pool can be deallocated by JVM because there is no reference
> to them. The solution is to protect pool chunks from GC by the strong map of
> ChunkCreator introduced by HBASE-18010. Will prepare the patch today.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18232) Add variable size chunks to the MSLAB
[ https://issues.apache.org/jira/browse/HBASE-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-18232: -- Assignee: Anastasia Braginsky > Add variable size chunks to the MSLAB > - > > Key: HBASE-18232 > URL: https://issues.apache.org/jira/browse/HBASE-18232 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-18232-V03.patch > > > Add possibility to create a variable size chunks of memory, so any cell (of > any size) can reside on a chunk. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18660) Remove duplicate code from the checkAndPut method in HTable
[ https://issues.apache.org/jira/browse/HBASE-18660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138041#comment-16138041 ] Hudson commented on HBASE-18660: FAILURE: Integrated in Jenkins build HBase-2.0 #380 (See [https://builds.apache.org/job/HBase-2.0/380/]) HBASE-18660 Remove duplicate code from the checkAndPut method in HTable (stack: rev 9c36c0c52f53bd365e73225315920a8712c64f62) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > Remove duplicate code from the checkAndPut method in HTable > --- > > Key: HBASE-18660 > URL: https://issues.apache.org/jira/browse/HBASE-18660 > Project: HBase > Issue Type: Task >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-18660.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18655) TestAsyncClusterAdminApi2 failing sometimes
[
https://issues.apache.org/jira/browse/HBASE-18655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138042#comment-16138042
]
Hudson commented on HBASE-18655:
FAILURE: Integrated in Jenkins build HBase-2.0 #380 (See
[https://builds.apache.org/job/HBase-2.0/380/])
HBASE-18655 TestAsyncClusterAdminApi2 failing sometimes (stack: rev
f25b529011a160d63c2ed05d5d5daa82899479ca)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncClusterAdminApi2.java
> TestAsyncClusterAdminApi2 failing sometimes
> ---
>
> Key: HBASE-18655
> URL: https://issues.apache.org/jira/browse/HBASE-18655
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Mike Drob
>Assignee: stack
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18655.master.001.patch
>
>
> Investigating a test failure seen on HBASE-12349
> git bisect shows me the following:
> {noformat}
> # first bad commit: [473446719b7b81b56216862bf2a94a576ff90f60] HBASE-18511
> Default no regions on master
> {noformat}
> It wouldn't fail every time, but I used this command with the bisect
> {noformat}
> export OPTS='test
> -Dtest=org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 -pl
> hbase-server -am'
> mvn clean $OPTS && mvn $OPTS && mvn $OPTS && mvn $OPTS
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138034#comment-16138034
]
Hudson commented on HBASE-18509:
FAILURE: Integrated in Jenkins build HBASE-14070.HLC #266 (See
[https://builds.apache.org/job/HBASE-14070.HLC/266/])
HBASE-18509 Encapsulate all clocks in RS/Master into a new 'Clocks' (stack: rev
4c56a3c28a5eea721f82dae841bb88c48a2bb04e)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClockWithCluster.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
* (add)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Clocks.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockNoopMasterServices.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionSplitPolicy.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALLockup.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - (done in 004.patch) --Maybe encapsulate all clocks in RS/Master into a new
> class. Then RSServices can just have getClocks() function.--
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18633) Add more info to understand the source/scenario of large batch requests exceeding threshold
[ https://issues.apache.org/jira/browse/HBASE-18633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138046#comment-16138046 ] Ted Yu commented on HBASE-18633: lgtm > Add more info to understand the source/scenario of large batch requests > exceeding threshold > --- > > Key: HBASE-18633 > URL: https://issues.apache.org/jira/browse/HBASE-18633 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.1 >Reporter: Vikas Vishwakarma >Assignee: Vikas Vishwakarma > Fix For: 3.0.0 > > Attachments: HBASE-18633.master.001.patch > > > In our controlled test env, we are seeing frequent Large batch operation > detected warnings (as implemented in HBASE-18023). > We are not running any client with large batch sizes on this test env, but we > start seeing these warnings after some runtime. Maybe it is caused due to > some error / retry scenario. Could also be related to Phoenix index updates > based on surrounding activity in the logs. Need to add more info like > table/region name and anything else that will enable debugging the source or > the scenario in which these warnings occur. > 2017-08-12 03:40:33,919 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7108 Client: xxx > 2017-08-12 03:40:34,476 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7096 Client: xxx > 2017-08-12 03:40:34,483 WARN [4,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7091 Client: xxx > 2017-08-12 03:40:35,728 WARN [3,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7102 Client: xxx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18633) Add more info to understand the source/scenario of large batch requests exceeding threshold
[ https://issues.apache.org/jira/browse/HBASE-18633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138066#comment-16138066 ] Abhishek Singh Chouhan commented on HBASE-18633: +1 pending QA > Add more info to understand the source/scenario of large batch requests > exceeding threshold > --- > > Key: HBASE-18633 > URL: https://issues.apache.org/jira/browse/HBASE-18633 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.1 >Reporter: Vikas Vishwakarma >Assignee: Vikas Vishwakarma > Fix For: 3.0.0 > > Attachments: HBASE-18633.master.001.patch > > > In our controlled test env, we are seeing frequent Large batch operation > detected warnings (as implemented in HBASE-18023). > We are not running any client with large batch sizes on this test env, but we > start seeing these warnings after some runtime. Maybe it is caused due to > some error / retry scenario. Could also be related to Phoenix index updates > based on surrounding activity in the logs. Need to add more info like > table/region name and anything else that will enable debugging the source or > the scenario in which these warnings occur. > 2017-08-12 03:40:33,919 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7108 Client: xxx > 2017-08-12 03:40:34,476 WARN [7,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7096 Client: xxx > 2017-08-12 03:40:34,483 WARN [4,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7091 Client: xxx > 2017-08-12 03:40:35,728 WARN [3,queue=0,port=16020] > regionserver.RSRpcServices - Large batch operation detected (greater than > 5000) (HBASE-18023). Requested Number of Rows: 7102 Client: xxx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18169) Coprocessor fix and cleanup before 2.0.0 release
[
https://issues.apache.org/jira/browse/HBASE-18169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138085#comment-16138085
]
Duo Zhang commented on HBASE-18169:
---
When revisiting HBASE-16320 I found that the {{Store}} and {{StoreFile}}
related CPs are totally messed up...
We have a {{preCompactScannerOpen}} and a {{preCompact}}, which are all aimed
to replace the source scanner which is used for compaction. But in fact, there
is no way to create a {{StoreScanner}} without touching the IA.Private classes.
So the only possible way seems to be wrapping the {{InternalScanner}} and doing
filtering in that class. I think it is acceptable for most cases? What do you
guys think? Do we still need to give user the ability to create a
{{StoreScanner}}? Or just cleanup our CP interfaces and tell user that you
should not try to create a {{StoreScanner}} any more, just wrap the given
{{InternalScanner}}?
We need to hurry up. This will be a big project, lots of API changes.
Thanks.
> Coprocessor fix and cleanup before 2.0.0 release
>
>
> Key: HBASE-18169
> URL: https://issues.apache.org/jira/browse/HBASE-18169
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 2.0.0-alpha-1
>Reporter: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-alpha-3
>
>
> As discussed in HBASE-18038. In RegionServerServices, Region and StoreFile
> interfaces we expose too many unnecessary methods. We need to find a way to
> not expose these methods to CP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18169) Coprocessor fix and cleanup before 2.0.0 release
[ https://issues.apache.org/jira/browse/HBASE-18169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138103#comment-16138103 ] Anoop Sam John commented on HBASE-18169: Creating new StoreFileScanner may be really not possible for user. So much internal.. Ya ideally wrapping is what mostly people wish to do. We can clearly document that. Only thing is the return type of the hook has to be a CP exposed interface type. This part we can make sure. Agree , we must clean all these before 2.0 > Coprocessor fix and cleanup before 2.0.0 release > > > Key: HBASE-18169 > URL: https://issues.apache.org/jira/browse/HBASE-18169 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 2.0.0-alpha-1 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-3 > > > As discussed in HBASE-18038. In RegionServerServices, Region and StoreFile > interfaces we expose too many unnecessary methods. We need to find a way to > not expose these methods to CP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18633) Add more info to understand the source/scenario of large batch requests exceeding threshold
[
https://issues.apache.org/jira/browse/HBASE-18633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138109#comment-16138109
]
Hadoop QA commented on HBASE-18633:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 30s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 43s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncTableScan |
| Timed out junit tests | org.apache.hadoop.hbase.coprocessor.TestHTableWrapper
|
| | org.apache.hadoop.hbase.client.TestAsyncReplicationAdminApi |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics |
| | org.apache.hadoop.hbase.TestZooKeeper |
| | org.apache.hadoop.hbase.client.TestAsyncQuotaAdminApi |
| | org.apache.hadoop.hbase.client.TestFromClientSide3 |
| | org.apache.hadoop.hbase.coprocessor.TestIncrementTimeRange |
| | org.apache.hadoop.hbase.TestClusterBootOrder |
| | org.apache.hadoop.hbase.TestJMXListener |
| | org.apache.hadoop.hbase.client.TestAvoidCellReferencesIntoShippedBlocks |
| | org.apache.hadoop.hbase.TestGlobalMemStoreSize |
| | org.apache.hadoop.hbase.client.TestSnapshotMetadata |
| | org.apache.hadoop.hbase.coprocessor.TestOpenTableInCoprocessor |
| | org.apache.hadoop.hbase.client.TestEnableTable |
| | org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient |
| | org.apache.hadoop.hbase.TestIOFencing |
| | org.apache.hadoop.hbase.zookeeper.TestZooKeeperACL |
| | org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient |
| |
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
|
| | org.apache.hadoop.hbase.snapshot.TestMobFlushSnapsho
[jira] [Commented] (HBASE-18655) TestAsyncClusterAdminApi2 failing sometimes
[
https://issues.apache.org/jira/browse/HBASE-18655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138112#comment-16138112
]
Hudson commented on HBASE-18655:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3581 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3581/])
HBASE-18655 TestAsyncClusterAdminApi2 failing sometimes (stack: rev
dcd3e9abf45352457715ff3ad24ca35d1dd9f2c2)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncClusterAdminApi2.java
> TestAsyncClusterAdminApi2 failing sometimes
> ---
>
> Key: HBASE-18655
> URL: https://issues.apache.org/jira/browse/HBASE-18655
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Mike Drob
>Assignee: stack
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-18655.master.001.patch
>
>
> Investigating a test failure seen on HBASE-12349
> git bisect shows me the following:
> {noformat}
> # first bad commit: [473446719b7b81b56216862bf2a94a576ff90f60] HBASE-18511
> Default no regions on master
> {noformat}
> It wouldn't fail every time, but I used this command with the bisect
> {noformat}
> export OPTS='test
> -Dtest=org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 -pl
> hbase-server -am'
> mvn clean $OPTS && mvn $OPTS && mvn $OPTS && mvn $OPTS
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18660) Remove duplicate code from the checkAndPut method in HTable
[ https://issues.apache.org/jira/browse/HBASE-18660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138111#comment-16138111 ] Hudson commented on HBASE-18660: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3581 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3581/]) HBASE-18660 Remove duplicate code from the checkAndPut method in HTable (stack: rev 7b8cf37c3b7cd82ba2e41947b64bf98bed121c55) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > Remove duplicate code from the checkAndPut method in HTable > --- > > Key: HBASE-18660 > URL: https://issues.apache.org/jira/browse/HBASE-18660 > Project: HBase > Issue Type: Task >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-18660.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18490) Modifying a table descriptor to enable replicas does not create replica regions
[ https://issues.apache.org/jira/browse/HBASE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138110#comment-16138110 ] ramkrishna.s.vasudevan commented on HBASE-18490: bq.So for branch-1, it is fine. Ya I agree. I have not seen the code in branch-1. The reference that you gave seems to be fine. Will work on a fix for 2.0 and above. > Modifying a table descriptor to enable replicas does not create replica > regions > --- > > Key: HBASE-18490 > URL: https://issues.apache.org/jira/browse/HBASE-18490 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0-alpha-1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: TestRegionReplicasWithRestartScenarios.java > > > After creating a table, if we try to modify the table to enable region > replication, the new Htable Descriptor is not taken into account and the > table is enabled again with default single region. > Ping [~enis], [~tedyu], [~devaraj]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18659) Use HDFS ACL to give user the ability to read snapshot directly on HDFS
[
https://issues.apache.org/jira/browse/HBASE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138114#comment-16138114
]
Anoop Sam John commented on HBASE-18659:
So whenever there is change for R permission on NS/Table/CF (Add or Remove)
have to do the same on all applicable HFiles
> Use HDFS ACL to give user the ability to read snapshot directly on HDFS
> ---
>
> Key: HBASE-18659
> URL: https://issues.apache.org/jira/browse/HBASE-18659
> Project: HBase
> Issue Type: New Feature
>Reporter: Duo Zhang
>
> On the dev meetup notes in Shenzhen after HBaseCon Asia, there is a topic
> about the permission to read hfiles on HDFS directly.
> {quote}
> For client-side scanner going against hfiles directly; is there a means of
> being able to pass the permissions from hbase to hdfs?
> {quote}
> And at Xiaomi we also face the same problem. {{SnapshotScanner}} is much
> faster and consumes less resources, but only super use has the ability to
> read hfile directly on HDFS.
> So here we want to use HDFS ACL to address this problem.
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_File_System_API
> The basic idea is to set acl and default on the table directory on HDFS for
> the users who have the permission to read the table on HBase.
> Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18659) Use HDFS ACL to give user the ability to read snapshot directly on HDFS
[
https://issues.apache.org/jira/browse/HBASE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138121#comment-16138121
]
Duo Zhang commented on HBASE-18659:
---
{quote}
So whenever there is change for R permission on NS/Table/CF (Add or Remove)
have to do the same on all applicable HFiles
{quote}
I'm not fully understand how the 'default ACL' works in HDFS. The semantic of
'default ACL' is that it will be applied to all the children of a directory,
but I do not know whether it will effect the already existing children, or only
newly created children.
> Use HDFS ACL to give user the ability to read snapshot directly on HDFS
> ---
>
> Key: HBASE-18659
> URL: https://issues.apache.org/jira/browse/HBASE-18659
> Project: HBase
> Issue Type: New Feature
>Reporter: Duo Zhang
>
> On the dev meetup notes in Shenzhen after HBaseCon Asia, there is a topic
> about the permission to read hfiles on HDFS directly.
> {quote}
> For client-side scanner going against hfiles directly; is there a means of
> being able to pass the permissions from hbase to hdfs?
> {quote}
> And at Xiaomi we also face the same problem. {{SnapshotScanner}} is much
> faster and consumes less resources, but only super use has the ability to
> read hfile directly on HDFS.
> So here we want to use HDFS ACL to address this problem.
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_File_System_API
> The basic idea is to set acl and default on the table directory on HDFS for
> the users who have the permission to read the table on HBase.
> Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18662) The default values for many configuration items in the code are not consistent with hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-18662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137963#comment-16137963
]
Yun Zhao edited comment on HBASE-18662 at 8/23/17 9:46 AM:
---
[~st...@apache.org] Need to keep the same as hbase-default.xml?
was (Author: yun zhao):
[~st...@duboce.net] Need to keep the same as hbase-default.xml?
> The default values for many configuration items in the code are not
> consistent with hbase-default.xml
> -
>
> Key: HBASE-18662
> URL: https://issues.apache.org/jira/browse/HBASE-18662
> Project: HBase
> Issue Type: Improvement
>Reporter: Yun Zhao
>Priority: Minor
>
> Such as
> {code}
> hbase.ipc.server.callqueue.handler.factor
> hbase.regionserver.logroll.errors.tolerated
> hbase.regionserver.region.split.policy
> zookeeper.session.timeout
> hbase.client.retries.number
> hbase.client.max.perserver.tasks
> hbase.client.keyvalue.maxsize
> hbase.normalizer.period
> hbase.hstore.blockingStoreFiles
> hbase.snapshot.restore.take.failsafe.snapshot
> hbase.lease.recovery.dfs.timeout
> hbase.rest.filter.classes
> hbase.http.max.threads
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[
https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138173#comment-16138173
]
Hadoop QA commented on HBASE-17614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 59 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
8s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: .
hbase-assembly {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}112m 16s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
31s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}156m
4s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}370m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 |
\\
\\
|| Subsystem || Report/Notes ||
| Docke
[jira] [Commented] (HBASE-18658) Purge hokey hbase Service implementation; use (internal) Guava Service instead
[
https://issues.apache.org/jira/browse/HBASE-18658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138018#comment-16138018
]
Hadoop QA commented on HBASE-18658:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.locking.TestLockManager |
| | hadoop.hbase.master.locking.TestLockProcedure |
| | hadoop.hbase.master.balancer.TestRegionLocationFinder |
| | hadoop.hbase.ipc.TestNettyRpcServer |
| | hadoop.hbase.regionserver.TestHRegionFileSystem |
| | hadoop.hbase.procedure.TestProcedureManager |
| | hadoop.hbase.backup.TestBackupHFileCleaner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18658 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883264/HBASE-18658.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux a38d6d5b9d07 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenki
[jira] [Commented] (HBASE-18224) Upgrade jetty
[
https://issues.apache.org/jira/browse/HBASE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138210#comment-16138210
]
Hadoop QA commented on HBASE-18224:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}142m 3s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}161m 36s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
51s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}376m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestInfoServers |
| | hadoop.hbase.master.TestGetInfoPort |
| | hadoop.hbase.TestInfoServers |
| | hadoop.hbase.master.TestGetInfoPort |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18224 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883261/HBASE-18224.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile xml |
| uname | Linux 597ab3cae9f7 3.13.0-116-generic #163-Ubunt
[jira] [Commented] (HBASE-16276) Inconsistency in Meta on cluster initialization and inability to assign regions on cluster restarts
[ https://issues.apache.org/jira/browse/HBASE-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138213#comment-16138213 ] baimo commented on HBASE-16276: --- hi,all. I met the same problem in  0.98.1+cdh5.1.5+94-1, how about the progress of this issue? Thanks ! > Inconsistency in Meta on cluster initialization and inability to assign > regions on cluster restarts > > > Key: HBASE-16276 > URL: https://issues.apache.org/jira/browse/HBASE-16276 > Project: HBase > Issue Type: Bug >Reporter: Joseph >Priority: Critical > > I have recently been experimenting with a cluster running on the latest > Master commit: 519f87f > I initialize a cluster of 11 RegionServers and 3 HMaster's and then create a > giant pre-split table using HBase Pe (~500 regions). I then kill the entire > cluster while the HBase Pe test is still writing to the cluster. When I > restart the cluster, a single RS will somehow always fail to open up any > regions. There will also be a large number of regions stuck in transition > with either Pending_Close or Failed_Close on a variety of RS's in the cluster. > Running Hbase hbck when I first initialized the cluster produces: > *** > HBaseFsck command line options: > Version: 2.0.0-fb10-SNAPSHOT > Number of live region servers: 12 > Number of dead region servers: 0 > Master: hbasectrl377.ash2.facebook.com,16000,1469225695984 > Number of backup masters: 2 > Average load: 0.1 > Number of requests: 0 > Number of regions: 2 > Number of regions in transition: 0 > ERROR: hbase:meta, replicaId 0 is found on more than one region. > ERROR: hbase:meta table is not consistent. Run HBCK with proper fix options > to fix hbase:meta inconsistency. Exiting... > Summary: > 2 inconsistencies detected. > Status: INCONSISTENT > *** > Finally in the restarted cluster. The Meta region is assigned a region, which > looks very different from normal region names. > "hbase:meta,,1" > as opposed to > "hbase:meta,,1.1588230740" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18663) netty-transport-native-epoll is not found using 2.0.0-alpha2
Miklos Csanady created HBASE-18663:
--
Summary: netty-transport-native-epoll is not found using
2.0.0-alpha2
Key: HBASE-18663
URL: https://issues.apache.org/jira/browse/HBASE-18663
Project: HBase
Issue Type: Bug
Reporter: Miklos Csanady
I am working on a Flume HBase sink module.
When I switch maven version number for hbase* dependencies from 2.0.0-alpha-1
to 2.0.0-alpha2 the build fails:
{code}
Stacktrace
java.io.IOException: Shutting down
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
at
org.apache.flume.sink.hbase2.TestHBase2Sink.setUpOnce(TestHBase2Sink.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterno netty-transport-native-epoll in
java.library.path
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
... 29 more
Caused by: java.lang.UnsatisfiedLinkError: failed to load the required native
library
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.Epoll.ensureAvailability(Epoll.java:78)
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:38)
at
org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:61)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:554)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:469)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
... 32 more
Caused by: java.lang.UnsatisfiedLinkError: no netty-transport-native-epoll in
ja
[jira] [Created] (HBASE-18664) netty-transport-native-epoll is not found using 2.0.0-alpha2
Miklos Csanady created HBASE-18664:
--
Summary: netty-transport-native-epoll is not found using
2.0.0-alpha2
Key: HBASE-18664
URL: https://issues.apache.org/jira/browse/HBASE-18664
Project: HBase
Issue Type: Bug
Reporter: Miklos Csanady
I am working on a Flume HBase sink module.
When I switch maven version number for hbase* dependencies from 2.0.0-alpha-1
to 2.0.0-alpha2 the build fails:
{code}
Stacktrace
java.io.IOException: Shutting down
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
at
org.apache.flume.sink.hbase2.TestHBase2Sink.setUpOnce(TestHBase2Sink.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterno netty-transport-native-epoll in
java.library.path
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
... 29 more
Caused by: java.lang.UnsatisfiedLinkError: failed to load the required native
library
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.Epoll.ensureAvailability(Epoll.java:78)
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:38)
at
org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:61)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:554)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:469)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
... 32 more
Caused by: java.lang.UnsatisfiedLinkError: no netty-transport-native-epoll in
ja
[jira] [Commented] (HBASE-18649) Deprecate KV Usage in MR to move to Cells in 3.0
[ https://issues.apache.org/jira/browse/HBASE-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138285#comment-16138285 ] ramkrishna.s.vasudevan commented on HBASE-18649: So while moving over to Cell - in all the Mappers we can allow Cell but can we safely assume that every cell is of type Extendedcell and so we can set up the mapper output class type to be of ExtendedCell? ExtendedCell has all the APIs for write(), getSerializedSize() etc. If we don't do this this we are have to again convert the cell to KV. Thoughts?? > Deprecate KV Usage in MR to move to Cells in 3.0 > > > Key: HBASE-18649 > URL: https://issues.apache.org/jira/browse/HBASE-18649 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 2.0.0-alpha-2 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 3.0.0, 2.1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18665) ReversedScannerCallable invokes getRegionLocations incorrectly
Peter Somogyi created HBASE-18665: - Summary: ReversedScannerCallable invokes getRegionLocations incorrectly Key: HBASE-18665 URL: https://issues.apache.org/jira/browse/HBASE-18665 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13 Reporter: Peter Somogyi Assignee: Peter Somogyi The behavior of ReversedScannerCallable#prepare [1] and ScannerCallable#prepare [2] methods differ how they call RpcRetryingCallerWithReadReplicas.getRegionLocations method. The reversed scanner uses the 'reload' parameter directly as the first argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, getConnection(), getTableName(), getRow()) - however, the forward scanner passes '!reload'. The getRegionLocations first parameter is 'useCache', the way we use it in ScannerCallable is the correct one. The same call can be found in ReversedScannerCallable#locateRegionsInRange [3] also without negating its value. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138415#comment-16138415 ] Biju Nair commented on HBASE-18532: --- [~tedyu], looks like the tests passed after re-attaching the patch? > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, HBASE-18532-1.1.2.PATCH, > HBASE-18532.PATCH, HBASE-18532-V1.0.PATCH, HBASE-18532-V2.0.PATCH, > L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18665) ReversedScannerCallable invokes getRegionLocations incorrectly
[ https://issues.apache.org/jira/browse/HBASE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138420#comment-16138420 ] Mike Drob commented on HBASE-18665: --- Good find, [~psomogyi]! Do you know what is the impact here? Performance? Correctness? > ReversedScannerCallable invokes getRegionLocations incorrectly > -- > > Key: HBASE-18665 > URL: https://issues.apache.org/jira/browse/HBASE-18665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > > The behavior of ReversedScannerCallable#prepare [1] and > ScannerCallable#prepare [2] methods differ how they call > RpcRetryingCallerWithReadReplicas.getRegionLocations method. > The reversed scanner uses the 'reload' parameter directly as the first > argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, > getConnection(), getTableName(), getRow()) - however, the forward scanner > passes '!reload'. The getRegionLocations first parameter is 'useCache', the > way we use it in ScannerCallable is the correct one. > The same call can be found in ReversedScannerCallable#locateRegionsInRange > [3] also without negating its value. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18659) Use HDFS ACL to give user the ability to read snapshot directly on HDFS
[
https://issues.apache.org/jira/browse/HBASE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138421#comment-16138421
]
Wei-Chiu Chuang commented on HBASE-18659:
-
Sounds like a pretty good proposal, and a valid use case of HDFS ACLs.
Things can get more complicated when you want to synchronize permissions
between different HFiles. You would also want to make sure the permission
exposed to HBase is in sync with other services, such as Hive or Impala, or
MapReduce.
Apache Sentry is a project where it provides a centralized authorization
management for these services. From a HDFS perspective, the authorization of a
file is delegated to Sentry, and Sentry returns a HDFS ACL that is equivalent
to Hive table permissions (RBAC).
> Use HDFS ACL to give user the ability to read snapshot directly on HDFS
> ---
>
> Key: HBASE-18659
> URL: https://issues.apache.org/jira/browse/HBASE-18659
> Project: HBase
> Issue Type: New Feature
>Reporter: Duo Zhang
>
> On the dev meetup notes in Shenzhen after HBaseCon Asia, there is a topic
> about the permission to read hfiles on HDFS directly.
> {quote}
> For client-side scanner going against hfiles directly; is there a means of
> being able to pass the permissions from hbase to hdfs?
> {quote}
> And at Xiaomi we also face the same problem. {{SnapshotScanner}} is much
> faster and consumes less resources, but only super use has the ability to
> read hfile directly on HDFS.
> So here we want to use HDFS ACL to address this problem.
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_File_System_API
> The basic idea is to set acl and default on the table directory on HDFS for
> the users who have the permission to read the table on HBase.
> Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Status: Open (was: Patch Available) > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138423#comment-16138423 ] Mike Drob commented on HBASE-18503: --- +1 > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Attachment: HBASE-18503.v4.patch rebase patch for HBASE-18103 > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch, HBASE-18503.v4.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Status: Patch Available (was: Open) > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch, HBASE-18503.v4.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18546) Always overwrite the TS for Append/Increment unless no existing cells are found
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18546:
---
Status: Open (was: Patch Available)
> Always overwrite the TS for Append/Increment unless no existing cells are
> found
> ---
>
> Key: HBASE-18546
> URL: https://issues.apache.org/jira/browse/HBASE-18546
> Project: HBase
> Issue Type: New Feature
> Components: API, Client
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Labels: incompatibleChange
> Fix For: 2.0.0
>
> Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch,
> HBASE-18546.v2.patch
>
>
> We don't pass the custom timestamp for Increment, and the increment's
> timestamp always be rewrite. Hence, user can't increment a cell with custom
> timestamp.
> {code:title=ProtobufUtil.java}
> if (values != null && values.size() > 0) {
> for (Cell cell: values) {
> valueBuilder.clear();
> valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap(
> cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength()));
> valueBuilder.setValue(UnsafeByteOperations.unsafeWrap(
> cell.getValueArray(), cell.getValueOffset(),
> cell.getValueLength()));
> if (cell.getTagsLength() > 0) {
>
> valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(),
> cell.getTagsOffset(), cell.getTagsLength()));
> }
> columnBuilder.addQualifierValue(valueBuilder.build());
> }
> }
> {code}
> In contrast to Increment, user can append the cell with custom timestamp. It
> would be better that make their behavior consistent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18546) Always overwrite the TS for Append/Increment unless no existing cells are found
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18546:
---
Status: Patch Available (was: Open)
> Always overwrite the TS for Append/Increment unless no existing cells are
> found
> ---
>
> Key: HBASE-18546
> URL: https://issues.apache.org/jira/browse/HBASE-18546
> Project: HBase
> Issue Type: New Feature
> Components: API, Client
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Labels: incompatibleChange
> Fix For: 2.0.0
>
> Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch,
> HBASE-18546.v2.patch, HBASE-18546.v3.patch
>
>
> We don't pass the custom timestamp for Increment, and the increment's
> timestamp always be rewrite. Hence, user can't increment a cell with custom
> timestamp.
> {code:title=ProtobufUtil.java}
> if (values != null && values.size() > 0) {
> for (Cell cell: values) {
> valueBuilder.clear();
> valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap(
> cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength()));
> valueBuilder.setValue(UnsafeByteOperations.unsafeWrap(
> cell.getValueArray(), cell.getValueOffset(),
> cell.getValueLength()));
> if (cell.getTagsLength() > 0) {
>
> valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(),
> cell.getTagsOffset(), cell.getTagsLength()));
> }
> columnBuilder.addQualifierValue(valueBuilder.build());
> }
> }
> {code}
> In contrast to Increment, user can append the cell with custom timestamp. It
> would be better that make their behavior consistent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18546) Always overwrite the TS for Append/Increment unless no existing cells are found
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18546:
---
Attachment: HBASE-18546.v3.patch
rebase patch
> Always overwrite the TS for Append/Increment unless no existing cells are
> found
> ---
>
> Key: HBASE-18546
> URL: https://issues.apache.org/jira/browse/HBASE-18546
> Project: HBase
> Issue Type: New Feature
> Components: API, Client
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Labels: incompatibleChange
> Fix For: 2.0.0
>
> Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch,
> HBASE-18546.v2.patch, HBASE-18546.v3.patch
>
>
> We don't pass the custom timestamp for Increment, and the increment's
> timestamp always be rewrite. Hence, user can't increment a cell with custom
> timestamp.
> {code:title=ProtobufUtil.java}
> if (values != null && values.size() > 0) {
> for (Cell cell: values) {
> valueBuilder.clear();
> valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap(
> cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength()));
> valueBuilder.setValue(UnsafeByteOperations.unsafeWrap(
> cell.getValueArray(), cell.getValueOffset(),
> cell.getValueLength()));
> if (cell.getTagsLength() > 0) {
>
> valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(),
> cell.getTagsOffset(), cell.getTagsLength()));
> }
> columnBuilder.addQualifierValue(valueBuilder.build());
> }
> }
> {code}
> In contrast to Increment, user can append the cell with custom timestamp. It
> would be better that make their behavior consistent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18665) ReversedScannerCallable invokes getRegionLocations incorrectly
[ https://issues.apache.org/jira/browse/HBASE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138448#comment-16138448 ] Peter Somogyi commented on HBASE-18665: --- I think it can cause both. Performance issue since it will try to relocate the region when we explicitly say don't force reload server location. Correctness when want to force reload of server location but we end up using cached locations. > ReversedScannerCallable invokes getRegionLocations incorrectly > -- > > Key: HBASE-18665 > URL: https://issues.apache.org/jira/browse/HBASE-18665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > > The behavior of ReversedScannerCallable#prepare [1] and > ScannerCallable#prepare [2] methods differ how they call > RpcRetryingCallerWithReadReplicas.getRegionLocations method. > The reversed scanner uses the 'reload' parameter directly as the first > argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, > getConnection(), getTableName(), getRow()) - however, the forward scanner > passes '!reload'. The getRegionLocations first parameter is 'useCache', the > way we use it in ScannerCallable is the correct one. > The same call can be found in ReversedScannerCallable#locateRegionsInRange > [3] also without negating its value. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138451#comment-16138451 ] Ted Yu commented on HBASE-18532: Integrated to branch-2 and master. Patch doesn't apply on branch-1 > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, HBASE-18532-1.1.2.PATCH, > HBASE-18532.PATCH, HBASE-18532-V1.0.PATCH, HBASE-18532-V2.0.PATCH, > L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18665) ReversedScannerCallable invokes getRegionLocations incorrectly
[ https://issues.apache.org/jira/browse/HBASE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18665: -- Description: The behavior of ReversedScannerCallable#prepare [1] and ScannerCallable#prepare [2] methods differ how they call RpcRetryingCallerWithReadReplicas.getRegionLocations method. The reversed scanner uses the 'reload' parameter directly as the first argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, getConnection(), getTableName(), getRow()) - however, the forward scanner passes '!reload'. The getRegionLocations first parameter is 'useCache', the way we use it in ScannerCallable is the correct one. The same call can be found in ReversedScannerCallable#locateRegionsInRange [3] also without negating its value. [1] https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L89-L90 [2] https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L139-L140 [3] https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L143-L144 was: The behavior of ReversedScannerCallable#prepare [1] and ScannerCallable#prepare [2] methods differ how they call RpcRetryingCallerWithReadReplicas.getRegionLocations method. The reversed scanner uses the 'reload' parameter directly as the first argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, getConnection(), getTableName(), getRow()) - however, the forward scanner passes '!reload'. The getRegionLocations first parameter is 'useCache', the way we use it in ScannerCallable is the correct one. The same call can be found in ReversedScannerCallable#locateRegionsInRange [3] also without negating its value. > ReversedScannerCallable invokes getRegionLocations incorrectly > -- > > Key: HBASE-18665 > URL: https://issues.apache.org/jira/browse/HBASE-18665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > > The behavior of ReversedScannerCallable#prepare [1] and > ScannerCallable#prepare [2] methods differ how they call > RpcRetryingCallerWithReadReplicas.getRegionLocations method. > The reversed scanner uses the 'reload' parameter directly as the first > argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, > getConnection(), getTableName(), getRow()) - however, the forward scanner > passes '!reload'. The getRegionLocations first parameter is 'useCache', the > way we use it in ScannerCallable is the correct one. > The same call can be found in ReversedScannerCallable#locateRegionsInRange > [3] also without negating its value. > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L89-L90 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L139-L140 > [3] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L143-L144 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Always overwrite the TS for Append/Increment unless no existing cells are found
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138486#comment-16138486
]
Ted Yu commented on HBASE-18546:
lgtm, pending QA
> Always overwrite the TS for Append/Increment unless no existing cells are
> found
> ---
>
> Key: HBASE-18546
> URL: https://issues.apache.org/jira/browse/HBASE-18546
> Project: HBase
> Issue Type: New Feature
> Components: API, Client
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Labels: incompatibleChange
> Fix For: 2.0.0
>
> Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch,
> HBASE-18546.v2.patch, HBASE-18546.v3.patch
>
>
> We don't pass the custom timestamp for Increment, and the increment's
> timestamp always be rewrite. Hence, user can't increment a cell with custom
> timestamp.
> {code:title=ProtobufUtil.java}
> if (values != null && values.size() > 0) {
> for (Cell cell: values) {
> valueBuilder.clear();
> valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap(
> cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength()));
> valueBuilder.setValue(UnsafeByteOperations.unsafeWrap(
> cell.getValueArray(), cell.getValueOffset(),
> cell.getValueLength()));
> if (cell.getTagsLength() > 0) {
>
> valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(),
> cell.getTagsOffset(), cell.getTagsLength()));
> }
> columnBuilder.addQualifierValue(valueBuilder.build());
> }
> }
> {code}
> In contrast to Increment, user can append the cell with custom timestamp. It
> would be better that make their behavior consistent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138502#comment-16138502 ] Biju Nair commented on HBASE-18532: --- There are changes in master which is not there in branch-1. Assuming that is causing the issue? > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, HBASE-18532-1.1.2.PATCH, > HBASE-18532.PATCH, HBASE-18532-V1.0.PATCH, HBASE-18532-V2.0.PATCH, > L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18224) Upgrade jetty
[ https://issues.apache.org/jira/browse/HBASE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18224: -- Attachment: HBASE-18224.master.004.patch > Upgrade jetty > - > > Key: HBASE-18224 > URL: https://issues.apache.org/jira/browse/HBASE-18224 > Project: HBase > Issue Type: Improvement > Components: dependencies >Reporter: Balazs Meszaros >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18224.branch-2.001 (1).patch, > HBASE-18224.branch-2.001.patch, HBASE-18224.branch-2.002 (1).patch, > HBASE-18224.branch-2.002.patch, HBASE-18224.branch-2.003.patch, > HBASE-18224.master.001.patch, HBASE-18224.master.002.patch, > HBASE-18224.master.003.patch, HBASE-18224.master.004.patch > > > Jetty can be updated to 9.4.6 and thrift can be updated to 0.10.0. I tried to > update them in HBASE-17898 but some unit tests failed, so created a sub-task > for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138562#comment-16138562 ] stack commented on HBASE-17614: --- Commit quick. The failed test was fixed last night in another issue. > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-17614-v1.patch, HBASE-17614-v2.patch, > HBASE-17614-v3.patch, HBASE-17614-v4.patch, HBASE-17614-v5.patch, > HBASE-17614-v5.patch > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18665) ReversedScannerCallable invokes getRegionLocations incorrectly
[ https://issues.apache.org/jira/browse/HBASE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138569#comment-16138569 ] Peter Somogyi commented on HBASE-18665: --- [~enis], [~stack]. Both of you have worked on ReversedScannerCallable class before. This change is going to be quite small. I was planning to write a test for it to verify RpcRetryingCallerWithReadReplicas.getRegionLocations is called with the correct parameters, however, this method is static and this makes the verification not trivial. Do I need to add test to such a small modification? > ReversedScannerCallable invokes getRegionLocations incorrectly > -- > > Key: HBASE-18665 > URL: https://issues.apache.org/jira/browse/HBASE-18665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > > The behavior of ReversedScannerCallable#prepare [1] and > ScannerCallable#prepare [2] methods differ how they call > RpcRetryingCallerWithReadReplicas.getRegionLocations method. > The reversed scanner uses the 'reload' parameter directly as the first > argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, > getConnection(), getTableName(), getRow()) - however, the forward scanner > passes '!reload'. The getRegionLocations first parameter is 'useCache', the > way we use it in ScannerCallable is the correct one. > The same call can be found in ReversedScannerCallable#locateRegionsInRange > [3] also without negating its value. > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L89-L90 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L139-L140 > [3] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L143-L144 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138583#comment-16138583 ] Josh Elser commented on HBASE-17614: Hahha, yup, I'm on it ;) > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-17614-v1.patch, HBASE-17614-v2.patch, > HBASE-17614-v3.patch, HBASE-17614-v4.patch, HBASE-17614-v5.patch, > HBASE-17614-v5.patch > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138585#comment-16138585
]
Chia-Ping Tsai commented on HBASE-18632:
Running 20 times, TestMultiParallel works well.
+1
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18632:
---
Component/s: test
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18665) ReversedScannerCallable invokes getRegionLocations incorrectly
[ https://issues.apache.org/jira/browse/HBASE-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138588#comment-16138588 ] stack commented on HBASE-18665: --- Nice find [~psomogyi] Can it be made non-static if only so you can test it? Regards whether you need to add a test, this looks like tricky stuff. A test would ensure we don't break this in future. If not possible, then we can live w/o. Thanks boss. > ReversedScannerCallable invokes getRegionLocations incorrectly > -- > > Key: HBASE-18665 > URL: https://issues.apache.org/jira/browse/HBASE-18665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13 >Reporter: Peter Somogyi >Assignee: Peter Somogyi > > The behavior of ReversedScannerCallable#prepare [1] and > ScannerCallable#prepare [2] methods differ how they call > RpcRetryingCallerWithReadReplicas.getRegionLocations method. > The reversed scanner uses the 'reload' parameter directly as the first > argument - RpcRetryingCallerWithReadReplicas.getRegionLocations(reload, id, > getConnection(), getTableName(), getRow()) - however, the forward scanner > passes '!reload'. The getRegionLocations first parameter is 'useCache', the > way we use it in ScannerCallable is the correct one. > The same call can be found in ReversedScannerCallable#locateRegionsInRange > [3] also without negating its value. > [1] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L89-L90 > [2] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L139-L140 > [3] > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java#L143-L144 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138585#comment-16138585
]
Chia-Ping Tsai edited comment on HBASE-18632 at 8/23/17 4:28 PM:
-
Running 20 times, TestMultiParallel works well with patch.
+1
was (Author: chia7712):
Running 20 times, TestMultiParallel works well.
+1
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18632:
---
Fix Version/s: 2.0.0
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18663) netty-transport-native-epoll is not found using 2.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-18663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138593#comment-16138593
]
stack commented on HBASE-18663:
---
[~mcsanady] You set the system property mentioned in this section?
http://hbase.apache.org/book.html#thirdparty
> netty-transport-native-epoll is not found using 2.0.0-alpha2
> -
>
> Key: HBASE-18663
> URL: https://issues.apache.org/jira/browse/HBASE-18663
> Project: HBase
> Issue Type: Bug
>Reporter: Miklos Csanady
>
> I am working on a Flume HBase sink module.
> When I switch maven version number for hbase* dependencies from 2.0.0-alpha-1
> to 2.0.0-alpha2 the build fails:
> {code}
> Stacktrace
> java.io.IOException: Shutting down
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at
> org.apache.flume.sink.hbase2.TestHBase2Sink.setUpOnce(TestHBase2Sink.java:85)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
> Caused by: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterno netty-transport-native-epoll in
> java.library.path
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
> ... 29 more
> Caused by: java.lang.UnsatisfiedLinkError: failed to load the required native
> library
> at
> org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.Epoll.ensureAvailability(Epoll.java:78)
> at
> org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:38)
> at
> org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:61)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:554)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:469)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConst
[jira] [Updated] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18632:
---
Fix Version/s: (was: 2.0.0)
2.0.0-alpha-3
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0-alpha-3
>
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18632:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Thanks for the review, Chia-ping.
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 3.0.0, 2.0.0-alpha-3
>
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18573) Update Append and Delete to use Mutation#getCellList(family)
[
https://issues.apache.org/jira/browse/HBASE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138622#comment-16138622
]
Xiang Li commented on HBASE-18573:
--
Hi [~Jan Hentschel], thanks for the reply! Got your idea, but I might have a
different opinion.
You mentioned
bq. As you see, there's only a single call to add(), which makes it safe to
assume that the size of the list will be 1
It might be not like that. add() could be called several times to an Append
object, to add several cells. similarly, addColumn() could also be called
several times to an Append object to add several families, qualifiers and
values.
{code}
Append a1 = new Append(row1);
a1.add(c1);
a1.add(c2);
a1.add(c3);
...
table1.append(a1);
{code}
So setting initial capacity to 1 is good when only adding one cell or one
family/qualifier/value to the Append object and make HTable process it, but
when adding multiple cells, initial capacity = 1 will inflate the backing array
more frequently than setting the initial capacity to a larger number.
It varies in different use scenarios. Does it make sense to you?
> Update Append and Delete to use Mutation#getCellList(family)
>
>
> Key: HBASE-18573
> URL: https://issues.apache.org/jira/browse/HBASE-18573
> Project: HBase
> Issue Type: Improvement
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18573.master.000.patch
>
>
> In addxxx() of Put and Increment, Mutation#getCellList(family) is called to
> get cell list from familyMap. But in the other 2 sub-class of Mutation:
> Append and Delete, the logic like Mutation#getCellList(family) is used, like
> {code}
> List list = familyMap.get(family);
> if(list == null) {
> list = new ArrayList<>(1);
> }
> {code}
> in
> {code}
> public Delete addColumn(byte [] family, byte [] qualifier, long timestamp)
> {code}
> of Delete
> We could make them to call Mutation#getCellList(family) to get better
> encapsulation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18573) Update Append and Delete to use Mutation#getCellList(family)
[
https://issues.apache.org/jira/browse/HBASE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138622#comment-16138622
]
Xiang Li edited comment on HBASE-18573 at 8/23/17 4:45 PM:
---
Hi [~Jan Hentschel], thanks for the reply! Got your idea, but I might have a
different opinion.
You mentioned
bq. As you see, there's only a single call to add(), which makes it safe to
assume that the size of the list will be 1
It might be not like that. add() could be called several times to an Append
object, to add several cells. similarly, addColumn() could also be called
several times to an Append object to add several families, qualifiers and
values.
{code}
Append a1 = new Append(row1);
a1.add(c1);
a1.add(c2);
a1.add(c3);
...
table1.append(a1);
{code}
So setting initial capacity to 1 is good when only adding one cell or one
family/qualifier/value to the Append object and make HTable process it, but
when adding multiple cells, initial capacity = 1 will inflate the backing array
more frequently than setting the initial capacity to a larger number.
It varies in different use scenarios. Does it make sense to you?
was (Author: water):
Hi [~Jan Hentschel], thanks for the reply! Got your idea, but I might have a
different opinion.
You mentioned
bq. As you see, there's only a single call to add(), which makes it safe to
assume that the size of the list will be 1
It might be not like that. add() could be called several times to an Append
object, to add several cells. similarly, addColumn() could also be called
several times to an Append object to add several families, qualifiers and
values.
{code}
Append a1 = new Append(row1);
a1.add(c1);
a1.add(c2);
a1.add(c3);
...
table1.append(a1);
{code}
So setting initial capacity to 1 is good when only adding one cell or one
family/qualifier/value to the Append object and make HTable process it, but
when adding multiple cells, initial capacity = 1 will inflate the backing array
more frequently than setting the initial capacity to a larger number.
It varies in different use scenarios. Does it make sense to you?
> Update Append and Delete to use Mutation#getCellList(family)
>
>
> Key: HBASE-18573
> URL: https://issues.apache.org/jira/browse/HBASE-18573
> Project: HBase
> Issue Type: Improvement
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18573.master.000.patch
>
>
> In addxxx() of Put and Increment, Mutation#getCellList(family) is called to
> get cell list from familyMap. But in the other 2 sub-class of Mutation:
> Append and Delete, the logic like Mutation#getCellList(family) is used, like
> {code}
> List list = familyMap.get(family);
> if(list == null) {
> list = new ArrayList<>(1);
> }
> {code}
> in
> {code}
> public Delete addColumn(byte [] family, byte [] qualifier, long timestamp)
> {code}
> of Delete
> We could make them to call Mutation#getCellList(family) to get better
> encapsulation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18573) Update Append and Delete to use Mutation#getCellList(family)
[
https://issues.apache.org/jira/browse/HBASE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138622#comment-16138622
]
Xiang Li edited comment on HBASE-18573 at 8/23/17 4:46 PM:
---
Hi [~Jan Hentschel], thanks for the reply! Got your idea, but I might have a
different opinion.
You mentioned
bq. As you see, there's only a single call to add(), which makes it safe to
assume that the size of the list will be 1
It might be not like that. add() could be called multiple times to an single
Append object, to add several cells. Similarly, addColumn() could also be
called multiple times to an Append object to add several families, qualifiers
and values.
{code}
Append a1 = new Append(row1);
a1.add(c1);
a1.add(c2);
a1.add(c3);
...
table1.append(a1);
{code}
So setting initial capacity to 1 is good when only adding one cell or one
family/qualifier/value to the Append object and make HTable process it, but
when adding multiple cells, initial capacity = 1 will inflate the backing array
more frequently than setting the initial capacity to a larger number.
It varies in different use scenarios. Does it make sense to you?
was (Author: water):
Hi [~Jan Hentschel], thanks for the reply! Got your idea, but I might have a
different opinion.
You mentioned
bq. As you see, there's only a single call to add(), which makes it safe to
assume that the size of the list will be 1
It might be not like that. add() could be called several times to an Append
object, to add several cells. similarly, addColumn() could also be called
several times to an Append object to add several families, qualifiers and
values.
{code}
Append a1 = new Append(row1);
a1.add(c1);
a1.add(c2);
a1.add(c3);
...
table1.append(a1);
{code}
So setting initial capacity to 1 is good when only adding one cell or one
family/qualifier/value to the Append object and make HTable process it, but
when adding multiple cells, initial capacity = 1 will inflate the backing array
more frequently than setting the initial capacity to a larger number.
It varies in different use scenarios. Does it make sense to you?
> Update Append and Delete to use Mutation#getCellList(family)
>
>
> Key: HBASE-18573
> URL: https://issues.apache.org/jira/browse/HBASE-18573
> Project: HBase
> Issue Type: Improvement
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18573.master.000.patch
>
>
> In addxxx() of Put and Increment, Mutation#getCellList(family) is called to
> get cell list from familyMap. But in the other 2 sub-class of Mutation:
> Append and Delete, the logic like Mutation#getCellList(family) is used, like
> {code}
> List list = familyMap.get(family);
> if(list == null) {
> list = new ArrayList<>(1);
> }
> {code}
> in
> {code}
> public Delete addColumn(byte [] family, byte [] qualifier, long timestamp)
> {code}
> of Delete
> We could make them to call Mutation#getCellList(family) to get better
> encapsulation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138628#comment-16138628 ] Josh Elser commented on HBASE-18353: [~vrodionov], thoughts? ^ > Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in > HBASE-14614 > --- > > Key: HBASE-18353 > URL: https://issues.apache.org/jira/browse/HBASE-18353 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch > > > HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a > half-implemented reopen of a region when a store file goes missing. > This JIRA tracks the work to fix/enable the test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17614: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for taking up this task, Vlad! Your keen eyes are appreciated as always, Stack. Thanks, Mike, too for the coordination ;) > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-17614-v1.patch, HBASE-17614-v2.patch, > HBASE-17614-v3.patch, HBASE-17614-v4.patch, HBASE-17614-v5.patch, > HBASE-17614-v5.patch > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-11996: --- Comment: was deleted (was: Is this fixed by HBASE-15583? If yes, we should close this.) > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18656) Address issues found by error-prone
[
https://issues.apache.org/jira/browse/HBASE-18656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138655#comment-16138655
]
Mike Drob commented on HBASE-18656:
---
Tried to start on this, ran into
https://github.com/google/error-prone/issues/716
> Address issues found by error-prone
> ---
>
> Key: HBASE-18656
> URL: https://issues.apache.org/jira/browse/HBASE-18656
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Mike Drob
> Labels: beginner
>
> We should address the new compilation errors found by running with
> {{-PerrorProne}}.
> Can convert this to a top-level task and add subtasks for modules if desired
> (in which case, link it back to parent issue HBASE-12187, please)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18575) [AMv2] Enable and fix TestRestartCluster#testRetainAssignmentOnRestart on master
[ https://issues.apache.org/jira/browse/HBASE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18575: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2 and master. Thanks [~uagashe] for the considered fix. > [AMv2] Enable and fix TestRestartCluster#testRetainAssignmentOnRestart on > master > > > Key: HBASE-18575 > URL: https://issues.apache.org/jira/browse/HBASE-18575 > Project: HBase > Issue Type: Bug > Components: amv2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Critical > Fix For: 2.0.0 > > Attachments: hbase-18575.master.001.patch, > hbase-18575.master.001.patch > > > * Test creates 3 node cluster (master + 2 RS) and creates 3 tables with 1 > region each. > * It then takes a snapshot of region assignments > * Shuts down the cluster and restarts with 4 nodes (2 RS on same ports, 1 RS > on port where master was running previously and master on new port) > * Takes snapshot of region assignments > * Expected behavior is regions will be retained on RS with appropriate ports > Debugging done so far shows that, when cluster is restarted meta is loaded > but meta has old entries (though the ports are same for RSs, start timestamps > are different). AssignmentManager#processofflineServersWithOnlineRegions() > finds that old RS with different timestamps are not online even though > regions assigned to them are online; triggering submitting SCP for each those > RSs. Subprocedure AssignProcedure of SCP has forceNewPlan set to true. Which > triggers re-assignment of regions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18614) Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated HBASE-18614:
--
Attachment: HBASE-18614-BRANCH-1.PATCH
Attaching patch for {{branch-1}}.
> Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
> --
>
> Key: HBASE-18614
> URL: https://issues.apache.org/jira/browse/HBASE-18614
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.1.2
>Reporter: Biju Nair
>Assignee: Biju Nair
> Fix For: 3.0.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18614-1.1.2.PATCH, HBASE-18614-BRANCH-1.PATCH,
> HBASE-18614.PATCH, HBASE-18614-V1.0.PATCH, HBASE-18614-V2.0.PATCH
>
>
> Enabling offheap cache and setting {{BUCKET_CACHE_COMBINED_KEY}} to {{false}}
> in site xml to make offheap cache a strict L2 cache to LRU cache, disables
> the L2 stats normally rendered on region server UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18656) Address issues found by error-prone
[
https://issues.apache.org/jira/browse/HBASE-18656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138662#comment-16138662
]
stack commented on HBASE-18656:
---
I think it super duper coolio that errorprone is (almost) working.
> Address issues found by error-prone
> ---
>
> Key: HBASE-18656
> URL: https://issues.apache.org/jira/browse/HBASE-18656
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Mike Drob
> Labels: beginner
>
> We should address the new compilation errors found by running with
> {{-PerrorProne}}.
> Can convert this to a top-level task and add subtasks for modules if desired
> (in which case, link it back to parent issue HBASE-12187, please)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138665#comment-16138665 ] stack commented on HBASE-11996: --- [~chia7712] How does HBASE-15583 fix this one sir? Thanks. > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18448) EndPoint example for refreshing HFiles for stores
[
https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138667#comment-16138667
]
Ajay Jadhav commented on HBASE-18448:
-
[~anoop.hbase]
public int getNumHFilesForRS(final HRegionServer rs, final TableName
tableName,
final byte[] family) {
int numHFiles = 0;
for (Region region : rs.getOnlineRegions(tableName)) {
numHFiles += region.getStore(family).getStorefilesCount();
}
return numHFiles;
}
Above we are getting the online regions based on tableName. Not sure, what is
missing?
> EndPoint example for refreshing HFiles for stores
> --
>
> Key: HBASE-18448
> URL: https://issues.apache.org/jira/browse/HBASE-18448
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Affects Versions: 2.0.0, 1.3.1
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-18448.branch-1.001.patch,
> HBASE-18448.branch-1.002.patch, HBASE-18448.branch-1.003.patch,
> HBASE-18448.branch-1.004.patch, HBASE-18448.branch-1.005.patch,
> HBASE-18448.branch-1.006.patch, HBASE-18448.branch-1.007.patch,
> HBASE-18448.master.001.patch, HBASE-18448.master.002.patch
>
>
> In the case where multiple HBase clusters are sharing a common rootDir, even
> after flushing the data from
> one cluster doesn't mean that other clusters (replicas) will automatically
> pick the new HFile. Through this patch,
> we are exposing the refresh HFiles API which when issued from a replica will
> update the in-memory file handle list
> with the newly added file.
> This allows replicas to be consistent with the data written through the
> primary cluster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18614) Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138679#comment-16138679
]
Hadoop QA commented on HBASE-18614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-18614 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18614 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883365/HBASE-18614-BRANCH-1.PATCH
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8265/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
> --
>
> Key: HBASE-18614
> URL: https://issues.apache.org/jira/browse/HBASE-18614
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.1.2
>Reporter: Biju Nair
>Assignee: Biju Nair
> Fix For: 3.0.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18614-1.1.2.PATCH, HBASE-18614-BRANCH-1.PATCH,
> HBASE-18614.PATCH, HBASE-18614-V1.0.PATCH, HBASE-18614-V2.0.PATCH
>
>
> Enabling offheap cache and setting {{BUCKET_CACHE_COMBINED_KEY}} to {{false}}
> in site xml to make offheap cache a strict L2 cache to LRU cache, disables
> the L2 stats normally rendered on region server UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18596) A hbase1 cluster should be able to replicate to a hbase2 cluster; verify
[ https://issues.apache.org/jira/browse/HBASE-18596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138717#comment-16138717 ] Esteban Gutierrez commented on HBASE-18596: --- Initial run from PE to replicate from HBase 1.2 to HBase 2.0 worked so far and replication from HBase 2.0 to HBase 1.2 also worked without any major issue. I will try to code this into a proper it test in IntegrationTestReplication. > A hbase1 cluster should be able to replicate to a hbase2 cluster; verify > > > Key: HBASE-18596 > URL: https://issues.apache.org/jira/browse/HBASE-18596 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: Esteban Gutierrez >Priority: Blocker > Fix For: 2.0.0-alpha-3 > > > From the mailing list thread "[DISCUSS] hbase-2.0.0 compatibility > expectations", [~esteban] asks: > bq. Should we add additional details around replication as well? for > instance, shall we consider a hbase-1.x cluster as a client for a hbase-2.x > cluster? > The latter should be a blocker. Verify it works. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[
https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138746#comment-16138746
]
Hadoop QA commented on HBASE-18503:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 21 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}126m
49s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}186m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18503 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883345/HBASE-18503.v4.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 9acddf9aba3a 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dcd3e9a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8262/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8262/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Change ***Util and Master to
[jira] [Commented] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138750#comment-16138750 ] Chia-Ping Tsai commented on HBASE-11996: Sorry [~stack], i mixed something up. BTW, TD can store byte array, so it is easy to save any serializable object in TD. We can introduce an new class (Creator?) to replace the owner for TD. > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18573) Update Append and Delete to use Mutation#getCellList(family)
[
https://issues.apache.org/jira/browse/HBASE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138764#comment-16138764
]
Jan Hentschel commented on HBASE-18573:
---
[~water] Got your point, but I think the problem appears at a different place.
The list inside *add* isn't exposed to Append directly. So, calling *add* on
Append multiple times isn't the problem. The old code didn't took the list in
the *familyMap* into account when adding the same family multiple times. At
this point it could happen that the backing list in the *familyMap* inflates
more frequently. Nevertheless, this problem is solved with your change.
> Update Append and Delete to use Mutation#getCellList(family)
>
>
> Key: HBASE-18573
> URL: https://issues.apache.org/jira/browse/HBASE-18573
> Project: HBase
> Issue Type: Improvement
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18573.master.000.patch
>
>
> In addxxx() of Put and Increment, Mutation#getCellList(family) is called to
> get cell list from familyMap. But in the other 2 sub-class of Mutation:
> Append and Delete, the logic like Mutation#getCellList(family) is used, like
> {code}
> List list = familyMap.get(family);
> if(list == null) {
> list = new ArrayList<>(1);
> }
> {code}
> in
> {code}
> public Delete addColumn(byte [] family, byte [] qualifier, long timestamp)
> {code}
> of Delete
> We could make them to call Mutation#getCellList(family) to get better
> encapsulation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18614) Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18614:
---
Resolution: Fixed
Fix Version/s: 1.5.0
1.4.0
Status: Resolved (was: Patch Available)
> Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
> --
>
> Key: HBASE-18614
> URL: https://issues.apache.org/jira/browse/HBASE-18614
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.1.2
>Reporter: Biju Nair
>Assignee: Biju Nair
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18614-1.1.2.PATCH, HBASE-18614-BRANCH-1.PATCH,
> HBASE-18614.PATCH, HBASE-18614-V1.0.PATCH, HBASE-18614-V2.0.PATCH
>
>
> Enabling offheap cache and setting {{BUCKET_CACHE_COMBINED_KEY}} to {{false}}
> in site xml to make offheap cache a strict L2 cache to LRU cache, disables
> the L2 stats normally rendered on region server UI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138769#comment-16138769 ] Chia-Ping Tsai commented on HBASE-17614: We committed a useless file "hbase-backup/.DS_Store". [~elserj] Would you please remove it ? > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-17614-v1.patch, HBASE-17614-v2.patch, > HBASE-17614-v3.patch, HBASE-17614-v4.patch, HBASE-17614-v5.patch, > HBASE-17614-v5.patch > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Always overwrite the TS for Append/Increment unless no existing cells are found
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138771#comment-16138771
]
Hadoop QA commented on HBASE-18546:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}127m
56s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}187m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18546 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883347/HBASE-18546.v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 235f1a49b1a7 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / dcd3e9a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8263/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8263/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Always overwrite the TS for
[jira] [Commented] (HBASE-18594) Release hbase-2.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-18594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138774#comment-16138774 ] Mike Drob commented on HBASE-18594: --- [~stack] - still need to update the branch-2 version to -alpha3-SNAPSHOT? you want a new issue for that? > Release hbase-2.0.0-alpha2 > -- > > Key: HBASE-18594 > URL: https://issues.apache.org/jira/browse/HBASE-18594 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-3 > > > Theme of this release is updated dependencies, purge of checked-in generated > sources (we now generate inline with build), and our move to depend on the > new hbase-thirdparty project. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Status: Open (was: Patch Available) > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch, HBASE-18503.v4.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138776#comment-16138776 ] stack commented on HBASE-11996: --- No worries. Yeah, you can store anything into HTD... Would be nice to have Creator. > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Always overwrite the TS for Append/Increment unless no existing cells are found
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138780#comment-16138780
]
Chia-Ping Tsai commented on HBASE-18546:
Will commit it tomorrow if no objections.
> Always overwrite the TS for Append/Increment unless no existing cells are
> found
> ---
>
> Key: HBASE-18546
> URL: https://issues.apache.org/jira/browse/HBASE-18546
> Project: HBase
> Issue Type: New Feature
> Components: API, Client
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Labels: incompatibleChange
> Fix For: 2.0.0
>
> Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch,
> HBASE-18546.v2.patch, HBASE-18546.v3.patch
>
>
> We don't pass the custom timestamp for Increment, and the increment's
> timestamp always be rewrite. Hence, user can't increment a cell with custom
> timestamp.
> {code:title=ProtobufUtil.java}
> if (values != null && values.size() > 0) {
> for (Cell cell: values) {
> valueBuilder.clear();
> valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap(
> cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength()));
> valueBuilder.setValue(UnsafeByteOperations.unsafeWrap(
> cell.getValueArray(), cell.getValueOffset(),
> cell.getValueLength()));
> if (cell.getTagsLength() > 0) {
>
> valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(),
> cell.getTagsOffset(), cell.getTagsLength()));
> }
> columnBuilder.addQualifierValue(valueBuilder.build());
> }
> }
> {code}
> In contrast to Increment, user can append the cell with custom timestamp. It
> would be better that make their behavior consistent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Attachment: HBASE-18503.v5.patch rebase patch for HBASE-17614 > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch, HBASE-18503.v4.patch, > HBASE-18503.v5.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Status: Patch Available (was: Open) > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > Attachments: HBASE-18503.v0.patch, HBASE-18503.v1.patch, > HBASE-18503.v1.patch, HBASE-18503.v2.patch, HBASE-18503.v2.patch, > HBASE-18503.v2.patch, HBASE-18503.v3.patch, HBASE-18503.v4.patch, > HBASE-18503.v5.patch > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils > # HMaster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138790#comment-16138790 ] Hudson commented on HBASE-18532: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3584 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3584/]) HBASE-18532 Improve cache related stats rendered on RS UI (tedyu: rev 04f114b85c0a44351556ad0e73f7e65f8e3051a4) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.java * (edit) hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java * (edit) hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/BlockCacheTmpl.jamon * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, HBASE-18532-1.1.2.PATCH, > HBASE-18532.PATCH, HBASE-18532-V1.0.PATCH, HBASE-18532-V2.0.PATCH, > L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18575) [AMv2] Enable and fix TestRestartCluster#testRetainAssignmentOnRestart on master
[ https://issues.apache.org/jira/browse/HBASE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138793#comment-16138793 ] Hudson commented on HBASE-18575: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3584 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3584/]) HBASE-18575 [AMv2] Fixed and enabled (stack: rev 6b21f8881be7649dadbdecd28dc2e2abe5c4ebe5) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredStochasticBalancer.java * (edit) hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRSGroupBasedLoadBalancer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/DistributedHBaseCluster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/favored/TestStartcodeAgnosticServerName.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeAssignmentHelper.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestFavoredStochasticLoadBalancer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestFavoredStochasticBalancerPickers.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeLoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/DeadServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodesPlan.java > [AMv2] Enable and fix TestRestartCluster#testRetainAssignmentOnRestart on > master > > > Key: HBASE-18575 > URL: https://issues.apache.org/jira/browse/HBASE-18575 > Project: HBase > Issue Type: Bug > Components: amv2 >Affects Versions: 2.0.0 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Critical > Fix For: 2.0.0 > > Attachments: hbase-18575.master.001.patch, > hbase-18575.master.001.patch > > > * Test creates 3 node cluster (master + 2 RS) and creates 3 tables with 1 > region each. > * It then takes a snapshot of region assignments > * Shuts down the cluster and restarts with 4 nodes (2 RS on same ports, 1 RS > on port where master was running previously and master on new port) > * Takes snapshot of region assignments > * Expected behavior is regions will be retained on RS with appropriate ports > Debugging done so far shows that, when cluster is restarted meta is loaded > but meta has old entries (though the ports are same for RSs, start timestamps > are different). AssignmentManager#processofflineServersWithOnlineRegions() > finds that old RS with different timestamps are not online even though > regions assigned to them are online; triggering submitting SCP for each those > RSs. Subprocedure AssignProcedure of SCP has forceNewPlan set to true. Which > triggers re-assignment of regions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18632) TestMultiParallel#testFlushCommitsWithAbort fails in master branch
[
https://issues.apache.org/jira/browse/HBASE-18632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138791#comment-16138791
]
Hudson commented on HBASE-18632:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3584 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3584/])
HBASE-18632 TestMultiParallel#testFlushCommitsWithAbort fails in master (tedyu:
rev 6c0e219dd42f766de345f3bbc991ea8900f0eb2f)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
> TestMultiParallel#testFlushCommitsWithAbort fails in master branch
> --
>
> Key: HBASE-18632
> URL: https://issues.apache.org/jira/browse/HBASE-18632
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 3.0.0, 2.0.0-alpha-3
>
> Attachments: 18632.v1.txt
>
>
> This can be reproduced:
> {code}
> java.lang.AssertionError: Waiting timed out after [15,000] msec
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.doTestFlushCommits(TestMultiParallel.java:310)
> at
> org.apache.hadoop.hbase.client.TestMultiParallel.testFlushCommitsWithAbort(TestMultiParallel.java:257)
> {code}
> The server count is affected by no-regions-on-master being the default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138792#comment-16138792 ] Hudson commented on HBASE-17614: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3584 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3584/]) HBASE-17614: Move Backup/Restore into separate module (Vladimir (elserj: rev 2dda371209b2e810fa76034b8fa8dcf47140e265) * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestFullBackup.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBoundaryTests.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupRepair.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupHFileCleaner.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestFullRestore.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/master/BackupLogCleaner.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupShowHistory.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/RestoreJob.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupMergeJob.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupMultipleDeletes.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/HBackupFileSystem.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalBackupManager.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/master/TestBackupLogCleaner.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/regionserver/LogRollRegionServerProcedureManager.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupTableInfo.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupHFileCleaner.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestRepairAfterFailedDelete.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupDriver.java * (add) hbase-backup/.DS_Store * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupException.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/TableBackupClient.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupDeleteWithFailures.java * (edit) hbase-it/pom.xml * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/util/BackupSet.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestFullBackupSetRestoreSet.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/RestoreRequest.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/RestoreJob.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/MapReduceBackupMergeJob.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestSystemTableSnapshot.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBoundaryTests.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupDeleteRestore.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupDriver.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestIncrementalBackupMergeWithFailures.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestFullBackupSet.java * (edit) hbase-assembly/src/main/assembly/hadoop-two-compat.xml * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/util/BackupUtils.java * (add) hbase-backup/pom.xml * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupObserver.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupAdminImpl.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupDelete.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalBackupManager.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalTableBackupClient.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/LogUtils.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupClientFactory.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupObserver.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupDescribe.java * (add) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestFullBackupWithFailures.java * (add) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupR
[jira] [Commented] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138799#comment-16138799 ] Chia-Ping Tsai commented on HBASE-11996: bq. Would be nice to have Creator. I can address this. [~mbertozzi] May i take this issue over? What fields should be in Creator? [~eclark] recommended the following fields # owner # email # webpage > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18594) Release hbase-2.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-18594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138802#comment-16138802 ] stack commented on HBASE-18594: --- Thanks [~mdrob] Pushed change as addendum on this list. > Release hbase-2.0.0-alpha2 > -- > > Key: HBASE-18594 > URL: https://issues.apache.org/jira/browse/HBASE-18594 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-3 > > > Theme of this release is updated dependencies, purge of checked-in generated > sources (we now generate inline with build), and our move to depend on the > new hbase-thirdparty project. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138810#comment-16138810 ] Josh Elser commented on HBASE-17614: bq. We committed a useless file "hbase-backup/.DS_Store". Josh Elser Would you please remove it ? Sorry about that! Just noticed [~appy] already removed it. Thanks! > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-17614-v1.patch, HBASE-17614-v2.patch, > HBASE-17614-v3.patch, HBASE-17614-v4.patch, HBASE-17614-v5.patch, > HBASE-17614-v5.patch > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18610) Provide capability to activate chaos monkey
[ https://issues.apache.org/jira/browse/HBASE-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18610: --- Attachment: 18610.v4.txt > Provide capability to activate chaos monkey > --- > > Key: HBASE-18610 > URL: https://issues.apache.org/jira/browse/HBASE-18610 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu > Attachments: 18610.v1.txt, 18610.v4.txt > > > Currently load-client runs against a cluster where region servers are stable. > We need to introduce chaos monkey so that wider coverage for read path is > exercised. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138813#comment-16138813 ] Appy commented on HBASE-17614: -- Removed the .DS_Store file. > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-17614-v1.patch, HBASE-17614-v2.patch, > HBASE-17614-v3.patch, HBASE-17614-v4.patch, HBASE-17614-v5.patch, > HBASE-17614-v5.patch > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18649) Deprecate KV Usage in MR to move to Cells in 3.0
[ https://issues.apache.org/jira/browse/HBASE-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138830#comment-16138830 ] stack commented on HBASE-18649: --- ExtendedCell is an internal class? @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC) The stuff in EC can't be in Cell... or we break everything. So we change ExtendedCell? But ExtendedCell has stuff in it like tags flag and getChunkId that we shouldn't expose? > Deprecate KV Usage in MR to move to Cells in 3.0 > > > Key: HBASE-18649 > URL: https://issues.apache.org/jira/browse/HBASE-18649 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 2.0.0-alpha-2 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 3.0.0, 2.1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18649) Deprecate KV Usage in MR to move to Cells in 3.0
[ https://issues.apache.org/jira/browse/HBASE-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138832#comment-16138832 ] stack commented on HBASE-18649: --- Oh, this is great by the way. > Deprecate KV Usage in MR to move to Cells in 3.0 > > > Key: HBASE-18649 > URL: https://issues.apache.org/jira/browse/HBASE-18649 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 2.0.0-alpha-2 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 3.0.0, 2.1.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138837#comment-16138837 ] stack commented on HBASE-11996: --- Sounds like a good start. Just take it [~chia7712] Mighty [~mbertozzi] not around these times > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16244) LocalHBaseCluster start timeout should be configurable
[
https://issues.apache.org/jira/browse/HBASE-16244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138860#comment-16138860
]
Andrey Elenskiy commented on HBASE-16244:
-
any reason this fix didn't make it to 1.3.x? We are hitting this in our
integration tests as well on 1.3.1.
> LocalHBaseCluster start timeout should be configurable
> --
>
> Key: HBASE-16244
> URL: https://issues.apache.org/jira/browse/HBASE-16244
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 1.0.1.1
>Reporter: Siddharth Wagle
> Fix For: 2.0.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16244.patch
>
>
> *Scenario*:
> - Ambari metrics service uses HBase in standalone mode
> - On restart of AMS HBase, the Master gives up in 30 seconds due to a
> hardcoded timeout in JVMClusterUtil
> {noformat}
> 2016-07-18 19:24:44,199 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Master not active after 30 seconds
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:194)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:445)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:227)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2526)
> {noformat}
> - On restart the current Master waits to become active and this leads to the
> timeout being triggered, waiting for a slightly longer time evades this issue.
> - The timeout it seems was meant for unit tests
> Attached patch allows the timeout to be configured via hbase-site as well as
> sets it to 5 minutes for clusters started through HMasterCommandLine.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18659) Use HDFS ACL to give user the ability to read snapshot directly on HDFS
[
https://issues.apache.org/jira/browse/HBASE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138891#comment-16138891
]
Andrew Purtell commented on HBASE-18659:
You can't do this only at the table level. ACLs can be applied at namespace,
table, column family levels, and also *per-cell*.
Setting aside cell ACLs as a special case, the AccessController has logic that
walks the hierarchy namespace -> table -> CF when doing permissions checks.
HDFS doesn't do this. Therefore all HDFS level ACLs must operate at the
smallest granularity, which is CF.
Setting HDFS level permissions on the CF which do not factor into account per
cell ACLs will break access to those cells granted special access. That said,
as an incompatible change for HBase 3, we can remove cell ACLs. Or, it could be
adequate to just document that scanning snapshots directly on HDFS is
incompatible with cell ACLs, so cell ACLs would be ignored.
> Use HDFS ACL to give user the ability to read snapshot directly on HDFS
> ---
>
> Key: HBASE-18659
> URL: https://issues.apache.org/jira/browse/HBASE-18659
> Project: HBase
> Issue Type: New Feature
>Reporter: Duo Zhang
>
> On the dev meetup notes in Shenzhen after HBaseCon Asia, there is a topic
> about the permission to read hfiles on HDFS directly.
> {quote}
> For client-side scanner going against hfiles directly; is there a means of
> being able to pass the permissions from hbase to hdfs?
> {quote}
> And at Xiaomi we also face the same problem. {{SnapshotScanner}} is much
> faster and consumes less resources, but only super use has the ability to
> read hfile directly on HDFS.
> So here we want to use HDFS ACL to address this problem.
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_File_System_API
> The basic idea is to set acl and default on the table directory on HDFS for
> the users who have the permission to read the table on HBase.
> Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18659) Use HDFS ACL to give user the ability to read snapshot directly on HDFS
[
https://issues.apache.org/jira/browse/HBASE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138892#comment-16138892
]
Andrew Purtell commented on HBASE-18659:
-1 on the notion of importing a third party access control solution, like
Sentry. We have access control in HBase already.
> Use HDFS ACL to give user the ability to read snapshot directly on HDFS
> ---
>
> Key: HBASE-18659
> URL: https://issues.apache.org/jira/browse/HBASE-18659
> Project: HBase
> Issue Type: New Feature
>Reporter: Duo Zhang
>
> On the dev meetup notes in Shenzhen after HBaseCon Asia, there is a topic
> about the permission to read hfiles on HDFS directly.
> {quote}
> For client-side scanner going against hfiles directly; is there a means of
> being able to pass the permissions from hbase to hdfs?
> {quote}
> And at Xiaomi we also face the same problem. {{SnapshotScanner}} is much
> faster and consumes less resources, but only super use has the ability to
> read hfile directly on HDFS.
> So here we want to use HDFS ACL to address this problem.
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_File_System_API
> The basic idea is to set acl and default on the table directory on HDFS for
> the users who have the permission to read the table on HBase.
> Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18628) ZKPermissionWatcher blocks all ZK notifications
[
https://issues.apache.org/jira/browse/HBASE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138907#comment-16138907
]
Andrew Purtell commented on HBASE-18628:
I'll review, and commit depending on that, the branch-1 patch to branch-1 and
branch-1.4 today
> ZKPermissionWatcher blocks all ZK notifications
> ---
>
> Key: HBASE-18628
> URL: https://issues.apache.org/jira/browse/HBASE-18628
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-3
>
> Attachments: HBASE-18628.branch-1.v5.patch, HBASE-18628.patch,
> HBASE-18628.v2.patch, HBASE-18628.v3.patch, HBASE-18628.v4.patch,
> HBASE-18628.v5.patch, HBASE-18628.v5.patch, jstack
>
>
> Buckle up folks, we're going for a ride here. I've seeing this on a branch-2
> based build, but I think the problem will affect branch-1 as well. I'm not
> able to easily reproduce the issue, but it will usually come up within an
> hour on a given cluster that I have, at which point the problem persists
> until an RS restart. I've been seeing the problem and paying attention for
> maybe two months, but I suspect it's been happening much longer than that.
> h3. Problem
> When running in a secure cluster, sometimes the ZK EventThread will get stuck
> on a permissions update and not be able to process new notifications. This
> happens to also block flush and snapshot, which is how we found it.
> h3. Analysis
> The main smoking gun is seeing this in repeated jstacks:
> {noformat}
> "main-EventThread" #43 daemon prio=5 os_prio=0 tid=0x7f0b92644000
> nid=0x6e69 waiting on condition [0x7f0b6730f000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.nodeChildrenChanged(ZKPermissionWatcher.java:191)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:503)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> {noformat}
> That sleep is a 20ms sleep in an {{AtomicReference.compareAndSet}} loop - but
> it never gets past the condition.
> {code}
> while (!nodes.compareAndSet(null, nodeList)) {
> try {
> Thread.sleep(20);
> } catch (InterruptedException e) {
> LOG.warn("Interrupted while setting node list", e);
> Thread.currentThread().interrupt();
> }
> }
> {code}
> The warning never shows up in the logs, it just keeps looping and looping.
> The last relevant line from the watcher in logs is:
> {noformat}
> 2017-08-17 21:25:12,379 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher:
> regionserver:22101-0x15df38884c80024, quorum=zk1:2181,zk2:2181,zk3:2181,
> baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged,
> state=SyncConnected, path=/hbase/acl
> {noformat}
> Which makes sense, because the code snippet is from permission watcher's
> {{nodeChildrenChanged}} handler.
> The separate thread introduced in HBASE-14370 is present, but not doing
> anything. And this event hasn't gotten to the part where it splits off into a
> thread:
> {noformat}
> "zk-permission-watcher4-thread-1" #160 daemon prio=5 os_prio=0
> tid=0x01750800 nid=0x6fd9 waiting on condition [0x7f0b5dce5000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007436ecea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> h3. Solutions
> There's a few approaches we can take to fix this, I think they are all
> complimentary. It might be useful to file subtasks or new issues for some of
> the solutions if they are longer term.
> # Move flush and snapshot to ProcedureV2. This makes my proximate problem go
> away, but it's only relevant to branch-2 and master, and doesn't fix any
