[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618590#comment-16618590 ] Duo Zhang commented on HBASE-21035: --- Theoretically there are changes, since the state has been changed to WAITING_TIMEOUT, and then back to RUNNABLE, and also the retrying count, the timeout value, etc. But is it necessary to persist these stuffs? For me, I do not think it is necessary to restore the WAITING_TIMEOUT state, if there is a crash and restart, then just starts like there is no failure before. For example, when we assign a region and fail, we change the procedure to WAITING_TIMEOUT state and want to sleep for 30 seconds. Then the master crashes and restarts, I think it is OK that we reschedule the TRSP immediately after restarting? What do you guys think? > Meta Table should be able to online even if all procedures are lost > --- > > Key: HBASE-21035 > URL: https://issues.apache.org/jira/browse/HBASE-21035 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.1.0 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Attachments: HBASE-21035.branch-2.0.001.patch, > HBASE-21035.branch-2.1.001.patch > > > After HBASE-20708, we changed the way we init after master starts. It will > only check WAL dirs and compare to Zookeeper RS nodes to decide which server > need to expire. For servers which's dir is ending with 'SPLITTING', we assure > that there will be a SCP for it. > But, if the server with the meta region crashed before master restarts, and > if all the procedure wals are lost (due to bug, or deleted manually, > whatever), the new restarted master will be stuck when initing. Since no one > will bring meta region online. > Although it is an anomaly case, but I think no matter what happens, we need > to online meta region. Otherwise, we are sitting ducks, noting can be done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21204) NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
[
https://issues.apache.org/jira/browse/HBASE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618695#comment-16618695
]
Hadoop QA commented on HBASE-21204:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 1 new + 19 unchanged
- 0 fixed = 20 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}202m
42s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}264m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21204 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940124/HBASE-21204.master.002.patch
|
| Optional Tests | asflicense cc unit hbaseprotoc javac javadoc find
[jira] [Created] (HBASE-21206) Scan with batch size may return incomplete cells
Zheng Hu created HBASE-21206: Summary: Scan with batch size may return incomplete cells Key: HBASE-21206 URL: https://issues.apache.org/jira/browse/HBASE-21206 Project: HBase Issue Type: Bug Reporter: Zheng Hu Assignee: Zheng Hu Attachments: ut.patch See the attached UT. the table has 5 columns and each column has at least one cell in it, but when we scan the table with batchSize=3, we only got 3 cells returned , the other 2 cells got lost ... It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21206: -- Priority: Critical (was: Major) > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Attachments: ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21205) 对时序数据的支持
[ https://issues.apache.org/jira/browse/HBASE-21205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618763#comment-16618763 ] Yu Li commented on HBASE-21205: --- Yes please use English although there's already some discussion about whether to allow other language. And we'd suggest to send questions like this to our user or dev [mailing list|http://hbase.apache.org/mail-lists.html] instead of creating JIRA. > 对时序数据的支持 > > > Key: HBASE-21205 > URL: https://issues.apache.org/jira/browse/HBASE-21205 > Project: HBase > Issue Type: New Feature >Reporter: lixiaobao >Priority: Minor > > 目前对于物联网的时序数据来说，每秒采集的数据点有几百万个点，每一行数据有几千列，数据写入hbase的时候比较缓慢，有没有计划以后添加对时序数据的支持？ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18822) Create table for peer cluster automatically when creating table in source cluster of using namespace replication.
[
https://issues.apache.org/jira/browse/HBASE-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618791#comment-16618791
]
Nihal Jain commented on HBASE-18822:
I tried this patch on my cluster and had the following observations/comments on
proposed patch:
* Table creation/alter ops are replicated to all its peer irrespective of the
replication state of the peers. It is good behavior wise as it will ensure
things are in sync completely. But, what if the admin disables a peer, do we
still want to sync create table to the peer? Or should we sync to a peer only
if the peer is enabled?
* Similarly table creation/alter ops are replicated irrespective of the
replication scope. It is good behavior wise as it will ensure things are in
sync completely. But, still we discuss about should we sync to a peer only if
the replication scope is 1?
* If table already exists in peer cluster but with a different descriptor, it
successfully creates table in active cluster. Should we log some warning if
table descriptor is different on the peer?
* If we perform the following ops in order:
** (a) Create table in active cluster
** (b) Add peer cluster
** (c) Modify the table created in step a
** *RESULT*: It will get stuck in procedure retry loop (in RUNNABLE STATE) and
will get a TableNotFoundException in each retry. The procedure will never
complete as it think it is a retriable error.
** *StackTrace:*
{noformat}
2018-09-17 22:37:46,840 WARN [PEWorker-6] procedure.ModifyTableProcedure:
Retriable error trying to modify table=t4 (in
state=MODIFY_TABLE_SYNC_SCHEMA_TO_PEER)
org.apache.hadoop.hbase.TableNotFoundException: t4
at
org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:548)
at
org.apache.hadoop.hbase.client.HBaseAdmin.getDescriptor(HBaseAdmin.java:338)
at
org.apache.hadoop.hbase.master.procedure.ModifyTableProcedure.syncSchemaModificationToPeer(ModifyTableProcedure.java:432)
at
org.apache.hadoop.hbase.master.procedure.ModifyTableProcedure.executeFromState(ModifyTableProcedure.java:132)
at
org.apache.hadoop.hbase.master.procedure.ModifyTableProcedure.executeFromState(ModifyTableProcedure.java:58)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:189)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:873)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1510)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1298)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1797)
{noformat}
* Table with the same name exists in active and peer cluster; both having
familiy c1. If we alter the table t1 and set version = 5 for family c1 in
active cluster, it will skip modifying table in peer. Because the patch only
handles supporting add column / delete column. Not to alter existing column
families. But by sync don't we mean we want to sync everything? What if the
active goes down with t1:c1:version=5, while the peer which is used for
disaster recovery solution just supports t1:c1:version=1 (i.e. whatever was
descriptor property of the column family while creation)
* Table with the same name exists in active and peer cluster but with families
f1, f2 in active and families c1, c2 in peer cluster. If we alter the table t1
to drop family f1 in active cluster, since the count logic is used (not
descriptor comparison. WHY?), it will replace the existing descriptor of peer
and drop families c1 and c2 (along with data, if any) from peer and add family
f1 to it (as this is the new descriptor in active).
* Table with the same name exists in active and peer cluster but with families
f1, f2 in active and family c1 in peer cluster. If we alter the table t1 to
drop family f1 in active cluster, it will skip modifying table in peer as count
is same. Again, should we log some warning if table descriptor is different on
the peer?
> Create table for peer cluster automatically when creating table in source
> cluster of using namespace replication.
> -
>
> Key: HBASE-18822
> URL: https://issues.apache.org/jira/browse/HBASE-18822
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0-alpha-2
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[
https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618875#comment-16618875
]
Hudson commented on HBASE-21102:
Results for branch branch-2
[build #1266 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1266/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1266//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1266//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1266//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ServerCrashProcedure should select target server where no other replicas
> exist for the current region
> -
>
> Key: HBASE-21102
> URL: https://issues.apache.org/jira/browse/HBASE-21102
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 3.0.0, 2.2.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Attachments: HBASE-21102_1.patch, HBASE-21102_2.patch,
> HBASE-21102_3.patch, HBASE-21102_4.patch, HBASE-21102_addendum.patch,
> HBASE-21102_addendum.patch, HBASE-21102_addendum.patch,
> HBASE-21102_branch-2.1.patch, HBASE-21102_branch-2.1.patch,
> HBASE-21102_initial.patch
>
>
> Currently when a server with region replica crashes, when the target server
> is created for the replica region assignment there is no guarentee that a
> server is selected where there is no other replica for the current region
> getting assigned. It so happens that currently we do an assignment randomly
> and later the LB comes and identifies these cases and again does MOVE for
> such regions. It will be better if we can identify target servers at least
> minimally ensuring that replicas are not colocated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost
[
https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618934#comment-16618934
]
Allan Yang commented on HBASE-21035:
{quote}
Then the master crashes and restarts, I think it is OK that we reschedule the
TRSP immediately after restarting? What do you guys think?
{quote}
Totally agree。
> Meta Table should be able to online even if all procedures are lost
> ---
>
> Key: HBASE-21035
> URL: https://issues.apache.org/jira/browse/HBASE-21035
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21035.branch-2.0.001.patch,
> HBASE-21035.branch-2.1.001.patch
>
>
> After HBASE-20708, we changed the way we init after master starts. It will
> only check WAL dirs and compare to Zookeeper RS nodes to decide which server
> need to expire. For servers which's dir is ending with 'SPLITTING', we assure
> that there will be a SCP for it.
> But, if the server with the meta region crashed before master restarts, and
> if all the procedure wals are lost (due to bug, or deleted manually,
> whatever), the new restarted master will be stuck when initing. Since no one
> will bring meta region online.
> Although it is an anomaly case, but I think no matter what happens, we need
> to online meta region. Otherwise, we are sitting ducks, noting can be done.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21207) Add client side sorting functionality in master web UI for table and region server details.
Archana Katiyar created HBASE-21207: --- Summary: Add client side sorting functionality in master web UI for table and region server details. Key: HBASE-21207 URL: https://issues.apache.org/jira/browse/HBASE-21207 Project: HBase Issue Type: Improvement Components: master, monitoring, UI, Usability Reporter: Archana Katiyar In Master UI, we can see region server details like requests per seconds and number of regions etc. Similarly, for tables also we can see online regions , offline regions. It will help ops people in determining hot spotting if we can provide sort functionality in the UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21207) Add client side sorting functionality in master web UI for table and region server details.
[ https://issues.apache.org/jira/browse/HBASE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Archana Katiyar updated HBASE-21207: Attachment: edc5c812-b928-11e8-87e2-ce6396629bbc.png 14926e82-b929-11e8-8bdd-4ce4621f1118.png 2724afd8-b929-11e8-8171-8b5b2ba3084e.png > Add client side sorting functionality in master web UI for table and region > server details. > --- > > Key: HBASE-21207 > URL: https://issues.apache.org/jira/browse/HBASE-21207 > Project: HBase > Issue Type: Improvement > Components: master, monitoring, UI, Usability >Reporter: Archana Katiyar >Priority: Minor > Attachments: 14926e82-b929-11e8-8bdd-4ce4621f1118.png, > 2724afd8-b929-11e8-8171-8b5b2ba3084e.png, > edc5c812-b928-11e8-87e2-ce6396629bbc.png > > > In Master UI, we can see region server details like requests per seconds and > number of regions etc. Similarly, for tables also we can see online regions , > offline regions. > It will help ops people in determining hot spotting if we can provide sort > functionality in the UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21207) Add client side sorting functionality in master web UI for table and region server details.
[ https://issues.apache.org/jira/browse/HBASE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618947#comment-16618947 ] Archana Katiyar commented on HBASE-21207: - We have patch available for this change. Attaching screenshot of the UI after this change. !2724afd8-b929-11e8-8171-8b5b2ba3084e.png! > Add client side sorting functionality in master web UI for table and region > server details. > --- > > Key: HBASE-21207 > URL: https://issues.apache.org/jira/browse/HBASE-21207 > Project: HBase > Issue Type: Improvement > Components: master, monitoring, UI, Usability >Reporter: Archana Katiyar >Priority: Minor > Attachments: 14926e82-b929-11e8-8bdd-4ce4621f1118.png, > 2724afd8-b929-11e8-8171-8b5b2ba3084e.png, > edc5c812-b928-11e8-87e2-ce6396629bbc.png > > > In Master UI, we can see region server details like requests per seconds and > number of regions etc. Similarly, for tables also we can see online regions , > offline regions. > It will help ops people in determining hot spotting if we can provide sort > functionality in the UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21207) Add client side sorting functionality in master web UI for table and region server details.
[ https://issues.apache.org/jira/browse/HBASE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618947#comment-16618947 ] Archana Katiyar edited comment on HBASE-21207 at 9/18/18 11:12 AM: --- We have patch available for this change. Attaching screenshot of the UI after this change. was (Author: archana.katiyar): We have patch available for this change. Attaching screenshot of the UI after this change. !2724afd8-b929-11e8-8171-8b5b2ba3084e.png! > Add client side sorting functionality in master web UI for table and region > server details. > --- > > Key: HBASE-21207 > URL: https://issues.apache.org/jira/browse/HBASE-21207 > Project: HBase > Issue Type: Improvement > Components: master, monitoring, UI, Usability >Reporter: Archana Katiyar >Priority: Minor > Attachments: 14926e82-b929-11e8-8bdd-4ce4621f1118.png, > 2724afd8-b929-11e8-8171-8b5b2ba3084e.png, > edc5c812-b928-11e8-87e2-ce6396629bbc.png > > > In Master UI, we can see region server details like requests per seconds and > number of regions etc. Similarly, for tables also we can see online regions , > offline regions. > It will help ops people in determining hot spotting if we can provide sort > functionality in the UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Attachment: HBASE-21206.v1.patch > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Attachments: HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Status: Patch Available (was: Open) > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Attachments: HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Fix Version/s: 2.0.3 2.1.1 2.2.0 > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Fix Version/s: 1.4.8 1.2.8 1.3.3 1.5.0 3.0.0 > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21035) Meta Table should be able to online even if all procedures are lost
[ https://issues.apache.org/jira/browse/HBASE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619097#comment-16619097 ] stack commented on HBASE-21035: --- So, a filter on state changes and we only persist critical info? Worth investigating, yes. > Meta Table should be able to online even if all procedures are lost > --- > > Key: HBASE-21035 > URL: https://issues.apache.org/jira/browse/HBASE-21035 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.1.0 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Attachments: HBASE-21035.branch-2.0.001.patch, > HBASE-21035.branch-2.1.001.patch > > > After HBASE-20708, we changed the way we init after master starts. It will > only check WAL dirs and compare to Zookeeper RS nodes to decide which server > need to expire. For servers which's dir is ending with 'SPLITTING', we assure > that there will be a SCP for it. > But, if the server with the meta region crashed before master restarts, and > if all the procedure wals are lost (due to bug, or deleted manually, > whatever), the new restarted master will be stuck when initing. Since no one > will bring meta region online. > Although it is an anomaly case, but I think no matter what happens, we need > to online meta region. Otherwise, we are sitting ducks, noting can be done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[
https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619112#comment-16619112
]
Hudson commented on HBASE-21102:
Results for branch master
[build #498 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/498/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/498//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/498//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/498//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ServerCrashProcedure should select target server where no other replicas
> exist for the current region
> -
>
> Key: HBASE-21102
> URL: https://issues.apache.org/jira/browse/HBASE-21102
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 3.0.0, 2.2.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Attachments: HBASE-21102_1.patch, HBASE-21102_2.patch,
> HBASE-21102_3.patch, HBASE-21102_4.patch, HBASE-21102_addendum.patch,
> HBASE-21102_addendum.patch, HBASE-21102_addendum.patch,
> HBASE-21102_branch-2.1.patch, HBASE-21102_branch-2.1.patch,
> HBASE-21102_initial.patch
>
>
> Currently when a server with region replica crashes, when the target server
> is created for the replica region assignment there is no guarentee that a
> server is selected where there is no other replica for the current region
> getting assigned. It so happens that currently we do an assignment randomly
> and later the LB comes and identifies these cases and again does MOVE for
> such regions. It will be better if we can identify target servers at least
> minimally ensuring that replicas are not colocated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21160) Assertion in TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels is ignored
[
https://issues.apache.org/jira/browse/HBASE-21160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619111#comment-16619111
]
Hudson commented on HBASE-21160:
Results for branch master
[build #498 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/498/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/498//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/498//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/498//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Assertion in
> TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels
> is ignored
> ---
>
> Key: HBASE-21160
> URL: https://issues.apache.org/jira/browse/HBASE-21160
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HBASE-21160-1.patch, HBASE-21160-2.patch,
> HBASE-21160-3.patch, HBASE-21160-4.patch, HBASE-21160-5.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14327/artifact/patchprocess/diff-compile-javac-hbase-server.txt
> (HBASE-21138 QA run):
> {code}
> [WARNING]
> /testptch/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDeletes.java:[315,25]
> [AssertionFailureIgnored] This assertion throws an AssertionError if it
> fails, which will be caught by an enclosing try block.
> {code}
> Here is related code:
> {code}
> PrivilegedExceptionAction scanAction = new
> PrivilegedExceptionAction() {
> @Override
> public Void run() throws Exception {
> try (Connection connection =
> ConnectionFactory.createConnection(conf);
> ...
> assertEquals(1, next.length);
> } catch (Throwable t) {
> throw new IOException(t);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Attachment: HBASE-21206.v1.patch > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21204) NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
[
https://issues.apache.org/jira/browse/HBASE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingyun Tian updated HBASE-21204:
-
Attachment: HBASE-21204.master.003.patch
> NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
>
>
> Key: HBASE-21204
> URL: https://issues.apache.org/jira/browse/HBASE-21204
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.1.0, 2.0.0, 2.2.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.1.0, 2.0.0, 2.2.0
>
> Attachments: HBASE-21204.master.001.patch,
> HBASE-21204.master.002.patch, HBASE-21204.master.003.patch
>
>
> There are 7 types of our Cell,
> Minimum((byte)0),
> Put((byte)4),
> Delete((byte)8),
> DeleteFamilyVersion((byte)10),
> DeleteColumn((byte)12),
> DeleteFamily((byte)14),
> Maximum((byte)255);
> But there are only 6 types of our CellType protobuf definition:
> enum CellType {
> MINIMUM = 0;
> PUT = 4;
> DELETE = 8;
> DELETE_FAMILY_VERSION = 10;
> DELETE_COLUMN = 12;
> DELETE_FAMILY = 14;
> MAXIMUM = 255;
> }
> Thus if we scan raw data which is DELETE_FAMILY_VERSION，it will throw NPE.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21206: Component/s: scan > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug > Components: scan >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619142#comment-16619142 ] Zheng Hu commented on HBASE-21206: -- Add some comment in the new patch.v1 to explain the corner case . As far as I know, only when the scan.nextRaw reach the batch limit and happen to exhaust all cells of the scan, the bug will occur... because response's result list and moreResultsInRegion come from two scan.nextRaw, it's a inconsistent state... Our online hbase user discovered the imcomplete cells, and we took few days to locate the bug. > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug > Components: scan >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21208) Bytes#toShort doesn't work without unsafe
Chia-Ping Tsai created HBASE-21208:
--
Summary: Bytes#toShort doesn't work without unsafe
Key: HBASE-21208
URL: https://issues.apache.org/jira/browse/HBASE-21208
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.2.0, 2.1.1, 2.0.3
seems we put the brackets in the wrong place :)
{code}
short n = 0;
n = (short) ((n ^ bytes[offset]) & 0xFF);
n = (short) (n << 8);
n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
return n;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-21208:
---
Description:
seems we put the brackets in the wrong place.
{code}
short n = 0;
n = (short) ((n ^ bytes[offset]) & 0xFF);
n = (short) (n << 8);
n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
return n;
{code}
was:
seems we put the brackets in the wrong place :)
{code}
short n = 0;
n = (short) ((n ^ bytes[offset]) & 0xFF);
n = (short) (n << 8);
n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
return n;
{code}
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-21208:
---
Attachment: HBASE-21208.v0.patch
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21208.v0.patch
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-21208:
---
Status: Patch Available (was: Open)
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21208.v0.patch
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[
https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619189#comment-16619189
]
Hadoop QA commented on HBASE-21206:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
39s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 39s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} hbase-server: The patch generated 0 new + 85
unchanged - 1 fixed = 85 total (was 86) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}125m
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21206 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940203/HBASE-21206.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux dd56b141695a 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/14441/artifact/patchprocess/patch-compile-hbase-server.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/14441/artifact/patchprocess/patch-compile-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14441/tes
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[ https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619212#comment-16619212 ] Mike Wingert commented on HBASE-21002: -- [~stack]: Sorry this took so long. I tried to follow what hbase proper was doing with packaging. I don't expect this to pass on the first try, I just need some feedback about direction. > Create assembly and scripts to start Kafka Proxy > > > Key: HBASE-21002 > URL: https://issues.apache.org/jira/browse/HBASE-21002 > Project: HBase > Issue Type: Sub-task > Components: hbase-connectors >Reporter: Mike Wingert >Assignee: Mike Wingert >Priority: Minor > > Add scripts for running and assembly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619229#comment-16619229
]
Hadoop QA commented on HBASE-21208:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} hbase-common: The patch generated 1 new + 111
unchanged - 0 fixed = 112 total (was 111) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 53s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
42s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940233/HBASE-21208.v0.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux eacada630f17 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/1/artifact/patchprocess/diff-checkstyle-hbase-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1/testReport/ |
| Max. process+thread count | 314 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://b
[jira] [Commented] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619230#comment-16619230
]
Ted Yu commented on HBASE-21208:
+1
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21208.v0.patch
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-21208:
---
Attachment: HBASE-21208.v1.patch
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21208.v0.patch, HBASE-21208.v1.patch
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619241#comment-16619241
]
Chia-Ping Tsai commented on HBASE-21208:
[~yuzhih...@gmail.com] Thanks for you reviews.
01
- fix checkstyle
- add a bit change to hbase-server so as to trigger full QA
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21208.v0.patch, HBASE-21208.v1.patch
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619241#comment-16619241
]
Chia-Ping Tsai edited comment on HBASE-21208 at 9/18/18 3:07 PM:
-
[~yuzhih...@gmail.com] Thanks for your reviews.
01
- fix checkstyle
- add a bit change to hbase-server so as to trigger full QA
was (Author: chia7712):
[~yuzhih...@gmail.com] Thanks for you reviews.
01
- fix checkstyle
- add a bit change to hbase-server so as to trigger full QA
> Bytes#toShort doesn't work without unsafe
> -
>
> Key: HBASE-21208
> URL: https://issues.apache.org/jira/browse/HBASE-21208
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21208.v0.patch, HBASE-21208.v1.patch
>
>
> seems we put the brackets in the wrong place.
> {code}
> short n = 0;
> n = (short) ((n ^ bytes[offset]) & 0xFF);
> n = (short) (n << 8);
> n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
> return n;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619268#comment-16619268 ] Ted Yu commented on HBASE-21206: Can you attach patch for branch-1 ? Thanks > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug > Components: scan >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619295#comment-16619295 ] Zheng Hu commented on HBASE-21206: -- bq. Can you attach patch for branch-1 ? Of course, I'll upload patch for each branch. Thanks. > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug > Components: scan >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Attachment: HBASE-21206.v2.patch > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug > Components: scan >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, > HBASE-21206.v2.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[
https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619381#comment-16619381
]
Hadoop QA commented on HBASE-21206:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
59s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 59s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 85
unchanged - 1 fixed = 85 total (was 86) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
45s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
11s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21206 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940227/HBASE-21206.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a08b380dcf3a 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| compile |
https://builds.apache.org/job/PreCommit-HBASE-Build/14442/artifact/patchprocess/patch-compile-hbase-server.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/14442/artifact/patchprocess/patch-compile-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14442/t
[jira] [Commented] (HBASE-21204) NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
[
https://issues.apache.org/jira/browse/HBASE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619404#comment-16619404
]
Hadoop QA commented on HBASE-21204:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 56s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
37s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}134m
52s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}199m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21204 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940229/HBASE-21204.master.003.patch
|
| Optional Tests | asflicense cc unit hbaseprotoc javac javadoc findbugs
shadedjars hadoopcheck hbaseanti checkstyle comp
[jira] [Commented] (HBASE-21208) Bytes#toShort doesn't work without unsafe
[
https://issues.apache.org/jira/browse/HBASE-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619565#comment-16619565
]
Hadoop QA commented on HBASE-21208:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 52s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
46s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}162m
2s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}218m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940240/HBASE-21208.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1d7a32662384 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| find
[jira] [Updated] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21156: -- Attachment: HBASE-21156.branch-2.1.003.patch > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21156: -- Attachment: HBASE-21156.branch-2.1.004.patch > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch, > HBASE-21156.branch-2.1.004.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619610#comment-16619610 ] stack commented on HBASE-21156: --- .004 Bug fix so hbck2 works against kerberized clusters. > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch, > HBASE-21156.branch-2.1.004.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-10342) RowKey Prefix Bloom Filter
[ https://issues.apache.org/jira/browse/HBASE-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619682#comment-16619682 ] Andrew Purtell commented on HBASE-10342: bq. Perhaps all that's needed is a "pre-bloom" coprocessor hook that can perform a transformation before a key is passed to the bloom filter (both read and write), or perhaps take over the entire hash calculation. That's an interesting idea. It could work. If you look at existing code for examples of how to make a coprocessor hook addition binary backwards compatible (won't be source compatible, that's fine) we could get it back to a version where it would be useful to Phoenix, 1.5.0 maybe, and 2.2.0. > RowKey Prefix Bloom Filter > -- > > Key: HBASE-10342 > URL: https://issues.apache.org/jira/browse/HBASE-10342 > Project: HBase > Issue Type: New Feature >Reporter: Liyin Tang >Priority: Major > > When designing HBase schema for some use cases, it is quite common to combine > multiple information within the RowKey. For instance, assuming that rowkey is > constructed as md5(id1) + id1 + id2, and user wants to scan all the rowkeys > which starting by id1. In such case, the rowkey bloom filter is able to cut > more unnecessary seeks during the scan. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[
https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619711#comment-16619711
]
Hadoop QA commented on HBASE-21206:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} hbase-server: The patch generated 0 new + 85
unchanged - 1 fixed = 85 total (was 86) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}272m 30s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}312m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.namespace.TestNamespaceAuditor |
| | hadoop.hbase.client.TestSnapshotTemporaryDirectory |
| | hadoop.hbase.util.TestFromClientSide3WoUnsafe |
| | hadoop.hbase.client.TestFromClientSide3 |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.client.TestCloneSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21206 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940244/HBASE-21206.v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9d17b161dd47 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbug
[jira] [Created] (HBASE-21209) Add HBaseInterfaceAudience HBCK to indicate that this tool is used by HBCK2?
stack created HBASE-21209: - Summary: Add HBaseInterfaceAudience HBCK to indicate that this tool is used by HBCK2? Key: HBASE-21209 URL: https://issues.apache.org/jira/browse/HBASE-21209 Project: HBase Issue Type: Sub-task Components: hbck2 Reporter: stack We have a new hbck Service and we are adding Procedures that hbck2 can use. Lets add a new audience designation hbck2 so can mark them apart (From [~Apache9] over in HBASE-21169) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21169) Initiate hbck2 tool in hbase-operator-tools repo
[ https://issues.apache.org/jira/browse/HBASE-21169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619781#comment-16619781 ] stack commented on HBASE-21169: --- bq. So lots of the recover code will still be in the hbase repo? Yes. A basic tenet is that Master does the repairs. bq. And HBCK2 is just a tool which asks HMaster to execute the recovery code? Yeah. Hopefully we can figure a few generic tools that can be combined variously to fix the sort of issues we could run into: finish-the-procedure, assigns, unassigns... bq. Maybe we need to introduce a new HBaseInterfaceAudience called HBCK to indicate that this tool is used by HBCK2? Makes sense. Adding. Filed HBASE-21209 > Initiate hbck2 tool in hbase-operator-tools repo > > > Key: HBASE-21169 > URL: https://issues.apache.org/jira/browse/HBASE-21169 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: Umesh Agashe >Assignee: stack >Priority: Major > Attachments: hbase-21169.master.001.patch > > > Create hbck2 tool in hbase-operator-tools > (https://github.com/apache/hbase-operator-tools.git) repo. This is not > intended to be complete tool but initial changes with usage, ability to > connect to server, logging, and using newly added HbckService etc. Code > changes to address specific use cases can be added later and tool will evolve. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21169) Initiate hbck2 tool in hbase-operator-tools repo
[
https://issues.apache.org/jira/browse/HBASE-21169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619789#comment-16619789
]
Umesh Agashe commented on HBASE-21169:
--
{quote}Maybe we need to introduce a new HBaseInterfaceAudience called HBCK to
indicate that this tool is used by HBCK2?
{quote}
Patch for HBASE-20941 already adds this. Its used as:
{code:java}
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.HBCK)
{code}
> Initiate hbck2 tool in hbase-operator-tools repo
>
>
> Key: HBASE-21169
> URL: https://issues.apache.org/jira/browse/HBASE-21169
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Affects Versions: 2.1.0
>Reporter: Umesh Agashe
>Assignee: stack
>Priority: Major
> Attachments: hbase-21169.master.001.patch
>
>
> Create hbck2 tool in hbase-operator-tools
> (https://github.com/apache/hbase-operator-tools.git) repo. This is not
> intended to be complete tool but initial changes with usage, ability to
> connect to server, logging, and using newly added HbckService etc. Code
> changes to address specific use cases can be added later and tool will evolve.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21192) Add HOW-TO repair damaged AMv2.
[
https://issues.apache.org/jira/browse/HBASE-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619791#comment-16619791
]
stack commented on HBASE-21192:
---
h2. STUCK Region Redux
The hbck2 tool can now do bulk assigning (HBASE-21156). On a cluster with 60k
regions stuck in the OPENING state (no locks held -- the OPENING state came
about because all MasterProcWALs had been removed from under a running
cluster), I did the following:
{code}
# First get list of all the STUCK and OPENING regions
$ grep STUCK master.log|grep OPENING|sed -e "s/^.*region=//"|sort -u >
/tmp/stuck.txt
# Split the file with 60k STUCK regions into files of 1k regions each.
$ split -l 1000 /tmp/stuck.txt STUCK
# Feed each file to the hbck2 tool... call assigns and pass list of 1k encoded
region names.
$ for i in `ls STUCK*`; do ls $i;
HBASE_CLASSPATH_PREFIX=./hbase-hbck2-1.0.0-SNAPSHOT.jar hbase
org.apache.hbase.HBCK2 assigns `cat $i|tr "\n" " "`; done
{code}
> Add HOW-TO repair damaged AMv2.
> ---
>
> Key: HBASE-21192
> URL: https://issues.apache.org/jira/browse/HBASE-21192
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> Need a page or two on how to do various fixups. Will include doc on how to
> identify particular circumstance, how to run a repair, as well as caveats
> (e.g. if no log recovery, then region may be missing edits).
> Add pointer to log messages, especially those that explicitly ask for
> operator intervention; e.g. Master#inMeta.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21169) Initiate hbck2 tool in hbase-operator-tools repo
[ https://issues.apache.org/jira/browse/HBASE-21169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619795#comment-16619795 ] stack commented on HBASE-21169: --- Thanks [~uagashe] Let me resolve. > Initiate hbck2 tool in hbase-operator-tools repo > > > Key: HBASE-21169 > URL: https://issues.apache.org/jira/browse/HBASE-21169 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: Umesh Agashe >Assignee: stack >Priority: Major > Attachments: hbase-21169.master.001.patch > > > Create hbck2 tool in hbase-operator-tools > (https://github.com/apache/hbase-operator-tools.git) repo. This is not > intended to be complete tool but initial changes with usage, ability to > connect to server, logging, and using newly added HbckService etc. Code > changes to address specific use cases can be added later and tool will evolve. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21209) Add HBaseInterfaceAudience HBCK to indicate that this tool is used by HBCK2?
[ https://issues.apache.org/jira/browse/HBASE-21209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-21209. --- Resolution: Duplicate HBASE-20941 already did this ([~uagashe]) > Add HBaseInterfaceAudience HBCK to indicate that this tool is used by HBCK2? > > > Key: HBASE-21209 > URL: https://issues.apache.org/jira/browse/HBASE-21209 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: stack >Priority: Major > > We have a new hbck Service and we are adding Procedures that hbck2 can use. > Lets add a new audience designation hbck2 so can mark them apart (From > [~Apache9] over in HBASE-21169) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619805#comment-16619805 ] stack commented on HBASE-21023: --- [~uagashe] is the new review up on rb addressed sir? Thanks. > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > Attachments: hbase-21023.master.001.patch, > hbase-21023.master.001.patch, hbase-21023.master.002.patch > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619807#comment-16619807 ] stack commented on HBASE-21156: --- Review please. This seems to work at scale. > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch, > HBASE-21156.branch-2.1.004.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619807#comment-16619807 ] stack edited comment on HBASE-21156 at 9/18/18 10:23 PM: - Review please. This seems to work at scale. See note at end of HBASE-21192. was (Author: stack): Review please. This seems to work at scale. > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch, > HBASE-21156.branch-2.1.004.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619828#comment-16619828 ] Umesh Agashe commented on HBASE-21023: -- ah! looking... > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > Attachments: hbase-21023.master.001.patch, > hbase-21023.master.001.patch, hbase-21023.master.002.patch > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619828#comment-16619828 ] Umesh Agashe edited comment on HBASE-21023 at 9/18/18 10:43 PM: ah! looking... Thanks [~stack]! was (Author: uagashe): ah! looking... > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > Attachments: hbase-21023.master.001.patch, > hbase-21023.master.001.patch, hbase-21023.master.002.patch > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21192) Add HOW-TO repair damaged AMv2.
[
https://issues.apache.org/jira/browse/HBASE-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619831#comment-16619831
]
stack commented on HBASE-21192:
---
To list outstanding locks and procedures:
{code}
$ echo "list_locks"| hbase shell &> /tmp/locks.txt
$ echo "list_procedures"| hbase shell &> /tmp/procedures.txt
{code}
> Add HOW-TO repair damaged AMv2.
> ---
>
> Key: HBASE-21192
> URL: https://issues.apache.org/jira/browse/HBASE-21192
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> Need a page or two on how to do various fixups. Will include doc on how to
> identify particular circumstance, how to run a repair, as well as caveats
> (e.g. if no log recovery, then region may be missing edits).
> Add pointer to log messages, especially those that explicitly ask for
> operator intervention; e.g. Master#inMeta.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21196) HTableMultiplexer clears the meta cache after every put operation
[
https://issues.apache.org/jira/browse/HBASE-21196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619836#comment-16619836
]
Ted Yu commented on HBASE-21196:
lgtm
> HTableMultiplexer clears the meta cache after every put operation
> -
>
> Key: HBASE-21196
> URL: https://issues.apache.org/jira/browse/HBASE-21196
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 3.0.0, 1.3.3, 2.2.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-21196.master.001.patch,
> HBASE-21196.master.001.patch, HBASE-21196.master.002.patch,
> HTableMultiplexer1000Puts.UT.txt
>
>
> *Problem:* Operations which use
> {{AsyncRequestFutureImpl.receiveMultiAction(MultiAction, ServerName,
> MultiResponse, int)}} API with tablename set to null reset the meta cache of
> the corresponding server after each call. One such operation is put operation
> of HTableMultiplexer (Might not be the only one). This may impact the
> performance of the system severely as all new ops directed to that server
> will have to go to zk first to get the meta table address and then get the
> location of the table region as it will become empty after every
> htablemultiplexer put.
> From the logs below, one can see after every other put the cached region
> locations are cleared. As a side effect of this, before every put the server
> needs to contact zk and get meta table location and read meta to get region
> locations of the table.
> {noformat}
> 2018-09-13 22:21:15,467 TRACE [htable-pool11-t1] client.MetaCache(283):
> Removed all cached region locations that map to
> root1-thinkpad-t440p,35811,1536857446588
> 2018-09-13 22:21:15,467 DEBUG [HTableFlushWorker-5]
> client.HTableMultiplexer$FlushWorker(632): Processed 1 put requests for
> root1-ThinkPad-T440p:35811 and 0 failed, latency for this send: 5
> 2018-09-13 22:21:15,515 TRACE
> [RpcServer.reader=1,bindAddress=root1-ThinkPad-T440p,port=35811]
> ipc.RpcServer$Connection(1954): RequestHeader call_id: 218 method_name: "Get"
> request_param: true priority: 0 timeout: 6 totalRequestSize: 137 bytes
> 2018-09-13 22:21:15,515 TRACE
> [RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=35811]
> ipc.CallRunner(105): callId: 218 service: ClientService methodName: Get size:
> 137 connection: 127.0.0.1:42338 executing as root1
> 2018-09-13 22:21:15,515 TRACE
> [RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=35811]
> ipc.RpcServer(2356): callId: 218 service: ClientService methodName: Get size:
> 137 connection: 127.0.0.1:42338 param: region=
> testHTableMultiplexer_1,,1536857451720.304d914b641a738624937c7f9b4d684f.,
> row=\x00\x00\x00\xC4 connection: 127.0.0.1:42338, response result {
> associated_cell_count: 1 stale: false } queueTime: 0 processingTime: 0
> totalTime: 0
> 2018-09-13 22:21:15,516 TRACE
> [RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=35811]
> io.BoundedByteBufferPool(106): runningAverage=16384, totalCapacity=0,
> count=0, allocations=1
> 2018-09-13 22:21:15,516 TRACE [main] ipc.AbstractRpcClient(236): Call: Get,
> callTime: 2ms
> 2018-09-13 22:21:15,516 TRACE [main] client.ClientScanner(122): Scan
> table=hbase:meta,
> startRow=testHTableMultiplexer_1,\x00\x00\x00\xC5,99
> 2018-09-13 22:21:15,516 TRACE [main] client.ClientSmallReversedScanner(179):
> Advancing internal small scanner to startKey at
> 'testHTableMultiplexer_1,\x00\x00\x00\xC5,99'
> 2018-09-13 22:21:15,517 TRACE [main] client.ZooKeeperRegistry(59): Looking up
> meta region location in ZK,
> connection=org.apache.hadoop.hbase.client.ZooKeeperRegistry@599f571f
> {noformat}
> From the minicluster logs [^HTableMultiplexer1000Puts.UT.txt] one can see
> that the string "Removed all cached region locations that map" and "Looking
> up meta region location in ZK" are present for every put.
> *Analysis:*
> The problem occurs as we call the {{cleanServerCache}} method always clears
> the server cache in case tablename is null and exception is null. See
> [AsyncRequestFutureImpl.java#L918|https://github.com/apache/hbase/blob/5d14c1af65c02f4e87059337c35e4431505de91c/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java#L918]
> {code:java}
> private void cleanServerCache(ServerName server, Throwable regionException) {
> if (tableName == null &&
> ClientExceptionsUtil.isMetaClearingException(regionException)) {
> // For multi-actions, we don't have a table name, but we want to make
> sure to clear the
> // cache in case there were location-related exceptions. We don't to
> clear the cache
> // for every possible exception that comes through, however.
> asyncProcess
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[
https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619866#comment-16619866
]
Hadoop QA commented on HBASE-21156:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
54s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
15s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 16m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} hbase-client: The patch generated 4 new + 211
unchanged - 0 fixed = 215 total (was 211) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
15s{color} | {color:red} hbase-server: The patch generated 3 new + 198
unchanged - 2 fixed = 201 total (was 200) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 40s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
58s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m
50s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}203m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:
[jira] [Updated] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21102: --- Attachment: 21102.addendum2.txt > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0, 2.2.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: 21102.addendum2.txt, HBASE-21102_1.patch, > HBASE-21102_2.patch, HBASE-21102_3.patch, HBASE-21102_4.patch, > HBASE-21102_addendum.patch, HBASE-21102_addendum.patch, > HBASE-21102_addendum.patch, HBASE-21102_branch-2.1.patch, > HBASE-21102_branch-2.1.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[
https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619886#comment-16619886
]
Ted Yu commented on HBASE-21102:
>From
>https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1267/testReport/junit/org.apache.hadoop.hbase.master.balancer/TestRSGroupBasedLoadBalancer/health_checks___yetus_jdk8_hadoop3_checks___testRetainAssignment/
> :
{code}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.balancer.TestRSGroupBasedLoadBalancer.testRetainAssignment(TestRSGroupBasedLoadBalancer.java:166)
{code}
It was due to this code:
{code}
if (this.services != null && this.services.getAssignmentManager() !=
null) { // for tests
if (!hasRegionReplica &&
this.services.getAssignmentManager().getRegionStates()
.isReplicaAvailableForRegion(region)) {
{code}
this.services.getAssignmentManager().getRegionStates() may return null.
Ram:
Can you take a look at 21102.addendum2.txt ?
The above test passes with the additional check.
> ServerCrashProcedure should select target server where no other replicas
> exist for the current region
> -
>
> Key: HBASE-21102
> URL: https://issues.apache.org/jira/browse/HBASE-21102
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 3.0.0, 2.2.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Attachments: 21102.addendum2.txt, HBASE-21102_1.patch,
> HBASE-21102_2.patch, HBASE-21102_3.patch, HBASE-21102_4.patch,
> HBASE-21102_addendum.patch, HBASE-21102_addendum.patch,
> HBASE-21102_addendum.patch, HBASE-21102_branch-2.1.patch,
> HBASE-21102_branch-2.1.patch, HBASE-21102_initial.patch
>
>
> Currently when a server with region replica crashes, when the target server
> is created for the replica region assignment there is no guarentee that a
> server is selected where there is no other replica for the current region
> getting assigned. It so happens that currently we do an assignment randomly
> and later the LB comes and identifies these cases and again does MOVE for
> such regions. It will be better if we can identify target servers at least
> minimally ensuring that replicas are not colocated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[
https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619904#comment-16619904
]
Hadoop QA commented on HBASE-21156:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
14s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
30s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 16m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} hbase-client: The patch generated 4 new + 211
unchanged - 0 fixed = 215 total (was 211) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
26s{color} | {color:red} hbase-server: The patch generated 3 new + 198
unchanged - 2 fixed = 201 total (was 200) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
4s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}174m
5s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}266m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:
[jira] [Updated] (HBASE-21206) Scan with batch size may return incomplete cells
[ https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21206: - Attachment: HBASE-21206.v3.patch > Scan with batch size may return incomplete cells > > > Key: HBASE-21206 > URL: https://issues.apache.org/jira/browse/HBASE-21206 > Project: HBase > Issue Type: Bug > Components: scan >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-21206.v1.patch, HBASE-21206.v1.patch, > HBASE-21206.v2.patch, HBASE-21206.v3.patch, ut.patch > > > See the attached UT. the table has 5 columns and each column has at least > one cell in it, but when we scan the table with batchSize=3, we only got 3 > cells returned , the other 2 cells got lost ... > It's a critial bug and should be fixed.. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-20704: Attachment: HBASE-20704.branch-1.001.patch > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch, > HBASE-20704.003.patch, HBASE-20704.004.draft.patch, HBASE-20704.005.patch, > HBASE-20704.006.patch, HBASE-20704.007.patch, HBASE-20704.branch-1.001.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619988#comment-16619988 ] Francis Liu commented on HBASE-20704: - [~apurtell] attached branch-1 patch. It's simpler but a bit different note that I had to backport a change in FSDataInputStreamWrapper so an IOException is thrown. Let me know if this looks good. > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch, > HBASE-20704.003.patch, HBASE-20704.004.draft.patch, HBASE-20704.005.patch, > HBASE-20704.006.patch, HBASE-20704.007.patch, HBASE-20704.branch-1.001.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21204) NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
[
https://issues.apache.org/jira/browse/HBASE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingyun Tian updated HBASE-21204:
-
Attachment: HBASE-21204.branch-2.001.patch
> NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
>
>
> Key: HBASE-21204
> URL: https://issues.apache.org/jira/browse/HBASE-21204
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.1.0, 2.0.0, 2.2.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 2.1.0, 2.0.0, 2.2.0
>
> Attachments: HBASE-21204.branch-2.001.patch,
> HBASE-21204.master.001.patch, HBASE-21204.master.002.patch,
> HBASE-21204.master.003.patch
>
>
> There are 7 types of our Cell,
> Minimum((byte)0),
> Put((byte)4),
> Delete((byte)8),
> DeleteFamilyVersion((byte)10),
> DeleteColumn((byte)12),
> DeleteFamily((byte)14),
> Maximum((byte)255);
> But there are only 6 types of our CellType protobuf definition:
> enum CellType {
> MINIMUM = 0;
> PUT = 4;
> DELETE = 8;
> DELETE_FAMILY_VERSION = 10;
> DELETE_COLUMN = 12;
> DELETE_FAMILY = 14;
> MAXIMUM = 255;
> }
> Thus if we scan raw data which is DELETE_FAMILY_VERSION，it will throw NPE.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[
https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620013#comment-16620013
]
Hadoop QA commented on HBASE-21102:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}123m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21102 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940320/21102.addendum2.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f8632bd77be4 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
|
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620019#comment-16620019
]
Reid Chan commented on HBASE-20993:
---
Thanks for the patch. [~jackbearden]
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 1.5.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFa
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 1.5.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedC
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620020#comment-16620020
]
Reid Chan commented on HBASE-20993:
---
Pushed to branch-1 and branch-1.4.
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 1.5.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFall
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Affects Version/s: 1.3.2
1.2.7
1.4.7
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6, 1.3.2, 1.2.7, 1.4.7
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 1.5.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(Kerbe
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Component/s: IPC/RPC
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 1.2.6, 1.3.2, 1.2.7, 1.4.7
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Fix For: 1.5.0, 1.4.8
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSExceptio
[jira] [Updated] (HBASE-21112) [Auth] IPC client fallback to simple auth (forward-port to master)
[ https://issues.apache.org/jira/browse/HBASE-21112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21112: -- Component/s: security IPC/RPC Client > [Auth] IPC client fallback to simple auth (forward-port to master) > -- > > Key: HBASE-21112 > URL: https://issues.apache.org/jira/browse/HBASE-21112 > Project: HBase > Issue Type: Bug > Components: Client, IPC/RPC, security >Affects Versions: 2.1.0, 2.0.2 >Reporter: Jack Bearden >Assignee: Jack Bearden >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21112) [Auth] IPC client fallback to simple auth (forward-port to master)
[ https://issues.apache.org/jira/browse/HBASE-21112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21112: -- Labels: (was: master) > [Auth] IPC client fallback to simple auth (forward-port to master) > -- > > Key: HBASE-21112 > URL: https://issues.apache.org/jira/browse/HBASE-21112 > Project: HBase > Issue Type: Bug > Components: Client, IPC/RPC, security >Affects Versions: 2.1.0, 2.0.2 >Reporter: Jack Bearden >Assignee: Jack Bearden >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21112) [Auth] IPC client fallback to simple auth (forward-port to master)
[ https://issues.apache.org/jira/browse/HBASE-21112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21112: -- Fix Version/s: 2.1.1 > [Auth] IPC client fallback to simple auth (forward-port to master) > -- > > Key: HBASE-21112 > URL: https://issues.apache.org/jira/browse/HBASE-21112 > Project: HBase > Issue Type: Bug > Components: Client, IPC/RPC, security >Affects Versions: 2.1.0, 2.0.2 >Reporter: Jack Bearden >Assignee: Jack Bearden >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21112) [Auth] IPC client fallback to simple auth (forward-port to master)
[ https://issues.apache.org/jira/browse/HBASE-21112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21112: -- Affects Version/s: 2.1.0 2.0.2 > [Auth] IPC client fallback to simple auth (forward-port to master) > -- > > Key: HBASE-21112 > URL: https://issues.apache.org/jira/browse/HBASE-21112 > Project: HBase > Issue Type: Bug > Components: Client, IPC/RPC, security >Affects Versions: 2.1.0, 2.0.2 >Reporter: Jack Bearden >Assignee: Jack Bearden >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-10342) RowKey Prefix Bloom Filter
[ https://issues.apache.org/jira/browse/HBASE-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620035#comment-16620035 ] Reid Chan commented on HBASE-10342: --- There's a patch available jira: HBASE-20636 > RowKey Prefix Bloom Filter > -- > > Key: HBASE-10342 > URL: https://issues.apache.org/jira/browse/HBASE-10342 > Project: HBase > Issue Type: New Feature >Reporter: Liyin Tang >Priority: Major > > When designing HBase schema for some use cases, it is quite common to combine > multiple information within the RowKey. For instance, assuming that rowkey is > constructed as md5(id1) + id1 + id2, and user wants to scan all the rowkeys > which starting by id1. In such case, the rowkey bloom filter is able to cut > more unnecessary seeks during the scan. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[
https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620083#comment-16620083
]
Hadoop QA commented on HBASE-20704:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} branch-1 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
44s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-1 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} hbase-server: The patch generated 0 new + 64
unchanged - 1 fixed = 64 total (was 65) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 37s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}104m
51s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-20704 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940338/HBASE-20704.branch-1.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 19a8c71cad8c 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 201
[jira] [Updated] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21023: -- Attachment: HBASE-21023.branch-2.1.001.patch > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > Attachments: HBASE-21023.branch-2.1.001.patch, > hbase-21023.master.001.patch, hbase-21023.master.001.patch, > hbase-21023.master.002.patch > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620085#comment-16620085 ] stack commented on HBASE-21023: --- Chatted w/ [~uagashe]. Addressed my own comments in this version of the patch and made it so the API can take multiple pids rather than do one at a time for the case where we want to do many procedures in the one invocation. > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: HBASE-21023.branch-2.1.001.patch, > hbase-21023.master.001.patch, hbase-21023.master.001.patch, > hbase-21023.master.002.patch > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21023) Add bypassProcedureToCompletion() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21023: -- Fix Version/s: 2.1.1 3.0.0 > Add bypassProcedureToCompletion() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: HBASE-21023.branch-2.1.001.patch, > hbase-21023.master.001.patch, hbase-21023.master.001.patch, > hbase-21023.master.002.patch > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19121) HBCK for AMv2 (A.K.A HBCK2)
[ https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620087#comment-16620087 ] stack commented on HBASE-19121: --- Hoisted up Design for HBCK2 from subissue. > HBCK for AMv2 (A.K.A HBCK2) > --- > > Key: HBASE-19121 > URL: https://issues.apache.org/jira/browse/HBASE-19121 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: stack >Assignee: Umesh Agashe >Priority: Major > Attachments: hbase-19121.master.001.patch > > > We don't have an hbck for the new AM. Old hbck may actually do damage going > against AMv2. > Fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21169) Initiate hbck2 tool in hbase-operator-tools repo
[ https://issues.apache.org/jira/browse/HBASE-21169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-21169. --- Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 3.0.0 I think its fair to say this has been well-started so resovling. I moved the 'design' from here up to the parent project. Its ongoing and it may get more eyeballs in the umbrella issue. > Initiate hbck2 tool in hbase-operator-tools repo > > > Key: HBASE-21169 > URL: https://issues.apache.org/jira/browse/HBASE-21169 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: Umesh Agashe >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: hbase-21169.master.001.patch > > > Create hbck2 tool in hbase-operator-tools > (https://github.com/apache/hbase-operator-tools.git) repo. This is not > intended to be complete tool but initial changes with usage, ability to > connect to server, logging, and using newly added HbckService etc. Code > changes to address specific use cases can be added later and tool will evolve. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21156: -- Attachment: HBASE-21156.branch-2.1.005.patch > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch, > HBASE-21156.branch-2.1.004.patch, HBASE-21156.branch-2.1.005.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21156) [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620092#comment-16620092 ] stack commented on HBASE-21156: --- .006 address checkstyle issues. > [hbck2] Queue an assign of hbase:meta and bulk assign/unassign > -- > > Key: HBASE-21156 > URL: https://issues.apache.org/jira/browse/HBASE-21156 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.1.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.1.1 > > Attachments: HBASE-21156.branch-2.1.001.patch, > HBASE-21156.branch-2.1.002.patch, HBASE-21156.branch-2.1.003.patch, > HBASE-21156.branch-2.1.004.patch, HBASE-21156.branch-2.1.005.patch > > > We need this to effect repair when damage. > If procedure WALs AND a server WAL dir are lost or cleaned or we crashed > during partial split (unlikely scenarios but nonetheless possible), a Master > can be stuck unable to become active because there is no assign procedure for > hbase:meta in the system. > The reasonable argument over in HBASE-21035 has it that attempts at > auto-repair under these extremes could cause other issues so at least until > we learn more, we for now punt to the operator for fix-up. > To reproduce the catastrophe, see notes in HBASE-21035 (and [~allan163]'s > test). > UPDATE: HBASE-21191 adds a Master assuming an "holding-pattern" if on startup > it does not have an assign for meta (possible if we lose all Master WAL > Procs.). Holding pattern is needed because we were exiting after one minute > of RPC'ing to old meta location. To inject an assign, the Admin#assign won't > work because it gets rejected because the "Master is Initializing". So we > need to be able to assign hbase:meta even if "Master is initializing". Also, > while in here, add being able to bulk assign because assigning a > Region-at-a-time from the shell only works if the offflined region count is > in the low 10s; fails when thousands offline. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620107#comment-16620107 ] ramkrishna.s.vasudevan commented on HBASE-21102: Thanks Ted. +1 for the addendum. Seems it is a 'SmallTest' and hence the RegionStates is null. But why is that the pre-commit did not indicate the test failure? And I can see it is not even there in the test report. Will you be committing the addendum? > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0, 2.2.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: 21102.addendum2.txt, HBASE-21102_1.patch, > HBASE-21102_2.patch, HBASE-21102_3.patch, HBASE-21102_4.patch, > HBASE-21102_addendum.patch, HBASE-21102_addendum.patch, > HBASE-21102_addendum.patch, HBASE-21102_branch-2.1.patch, > HBASE-21102_branch-2.1.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors
saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218108464 ## File path: kafka/README ## @@ -0,0 +1,126 @@ +Hbase Kafka Proxy Review comment: Should this be in the README.md? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218126550
##
File path: kafka/README
##
@@ -0,0 +1,126 @@
+Hbase Kafka Proxy
+
+Welcome to the hbase kafka proxy. The purpose of this proxy is to act as a
'fake peer'. It
+receives replication events from it's peer and applies a set of rules (stored
in
+kafka-route-rules.xml) to determine if the event should be forwared to a
topic. If the
+mutation matches one of the rules, the mutation is converted to an avro object
and the
+item is placed into the topic.
+
+The service sets up a bare bones region server, so it will use the values in
hbase-site.xml. If
+you wish to override those values, pass them as -Dkey=value.
+
+To Use:
+
+1. Make sure the hbase command is in your path. The proxy uses the 'hbase
classpath' command to
+find the hbase libraries.
+
+2. Create any topics in your kafka broker that you wish to use.
+
+3. set up kafka-route-rules.xml. This file controls how the mutations are
routed. There are
+two kinds of rules: route and drop. drop: any mutation that matches this rule
will be dropped.
+route: any mutation that matches this rule will be routed to the configured
topic.
+
+Each rule has the following parameters:
+- table
+- columnFamily
+- qualifier
+
+The qualifier parameter can contain simple wildcard expressions (start and end
only).
+
+Examples
+
+
+
+
+
+
+This causes all mutations done to default:mytable to be routed to kafka topic
'foo'
+
+
+
+
+
+
+This will cause all mutations from default:mytable columnFamily mycf and
qualifier myqualifier
+to be routed to mykafkatopic.
+
+
+
+
+
+
+
+This combination will route all mutations from default:mytable columnFamily
mycf to mykafkatopic
+unless they start with 'secret'. Items matching that rule will be dropped.
The way the rule is
+written, all other mutations for column family mycf will be routed to the
'mykafka' topic.
+
+4. Service arguments
+
+--kafkabrokers (or -b)
+--routerulesfile (or -r)
+--kafkaproperties (or -f)
+--peername (or -p) name of hbase peer to use (defaults to hbasekafka)
+--znode (or -z) root znode (defaults to /kafkaproxy)
+--enablepeer (or -e) enable peer on startup (defaults to false)]
+--auto (or -a) auto create peer
+
+
+5. start the service.
+ - make sure the hbase command is in your path
+ - ny default, the service looks for route-rules.xml in the conf directory,
you can specify a
+ differeent file or location with the -r argument
+
+bin/hbase-connectors-daemon.sh start kafkaproxy -a -e -p wootman -b
localhost:9092 -r ~/kafka-route-rules.xml
+
+this:
+- starts the kafka proxy
+- passes the -a. The proxy will create the replication peer specified by -p
if it does not exist
+ (not required, but it savecs some busy work).
+- enables the peer (-e) the proxy will enable the peer when the service starts
(not required, you can
+ manually enable the peer in the hbase shell)
+
+
+Notes:
+1. The proxy will connect to the zookeeper in hbase-site.xml by default. You
can override this by
+ passing -Dhbase.zookeeper.quorum
+
+ bin/hbase-connectors-daemon.sh start kafkaproxy
-Dhbase.zookeeper.quorum=localhost:1234 . other args
+
+2. route rules only support unicode characters.
+3. I do not have access to a secured hadoop clsuter to test this on.
+
+Message format
+
+Messages are in avro format, this is the schema:
+
+{"namespace": "org.apache.hadoop.hbase.kafka",
+ "type": "record",
+ "name": "HBaseKafkaEvent",
+ "fields": [
+{"name": "key", "type": "bytes"},
+{"name": "timestamp", "type": "long" },
+{"name": "delete", "type": "boolean" },
+{"name": "value", "type": "bytes"},
+{"name": "qualifier", "type": "bytes"},
+{"name": "family", "type": "bytes"},
+{"name": "table", "type": "bytes"}
+ ]
+}
+
+Any language that supports Avro should be able to consume the messages off the
topic.
+
+
+Testing Utility
+
+A utility is included to test the routing rules.
+
+bin/hbase-connectors-daemon.sh start kafkaproxytest -k -t
+
+the messages will be dumped in string format under logs/
+
+TODO:
+1. Some properties passed into the region server are hard-coded.
+2. The avro objects should be generic
+3. Allow rules to be refreshed without a restart
+4. Get this tested on a secure (TLS & Kerberos) enabled cluster.
Review comment:
this is great.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[
https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620114#comment-16620114
]
ASF GitHub Bot commented on HBASE-21002:
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218131154
##
File path: pom.xml
##
@@ -167,21 +170,34 @@
hbase-client
${hbase.version}
-
-org.apache.hbase.connectors
-hbase-kafka-model
-${project.version}
-
-
-org.apache.hbase.connectors
-hbase-kafka-proxy
-${project.version}
-
+
+
+
+ org.apache.hbase.connectors
+ hbase-kafka-proxy
+ ${project.version}
+
+
+
+ org.apache.hbase.connectors
+ hbase-kafka-model
+ ${project.version}
+
+
+
+
+ maven-assembly-plugin
+
+
+true
+
+
org.apache.maven.plugins
maven-release-plugin
Review comment:
This is great. Will merge in a while. Leaving it open another while in case
other comments.
I suppose we could change the model for how we do our scripts perhaps
bringing in the new hadoop3 mode but that we can do in a follow-on. This is a
very nice start.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Create assembly and scripts to start Kafka Proxy
>
>
> Key: HBASE-21002
> URL: https://issues.apache.org/jira/browse/HBASE-21002
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
>Reporter: Mike Wingert
>Assignee: Mike Wingert
>Priority: Minor
>
> Add scripts for running and assembly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218131154
##
File path: pom.xml
##
@@ -167,21 +170,34 @@
hbase-client
${hbase.version}
-
-org.apache.hbase.connectors
-hbase-kafka-model
-${project.version}
-
-
-org.apache.hbase.connectors
-hbase-kafka-proxy
-${project.version}
-
+
+
+
+ org.apache.hbase.connectors
+ hbase-kafka-proxy
+ ${project.version}
+
+
+
+ org.apache.hbase.connectors
+ hbase-kafka-model
+ ${project.version}
+
+
+
+
+ maven-assembly-plugin
+
+
+true
+
+
org.apache.maven.plugins
maven-release-plugin
Review comment:
This is great. Will merge in a while. Leaving it open another while in case
other comments.
I suppose we could change the model for how we do our scripts perhaps
bringing in the new hadoop3 mode but that we can do in a follow-on. This is a
very nice start.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[
https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620112#comment-16620112
]
ASF GitHub Bot commented on HBASE-21002:
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218127793
##
File path:
kafka/hbase-kafka-proxy/src/main/java/org/apache/hadoop/hbase/kafka/KafkaProxy.java
##
@@ -154,19 +150,28 @@ public static void main(String[] args) throws Exception {
options.addOption("e", "enablepeer", false,
"enable peer on startup (defaults to false)");
-
LOG.info("STARTING executorService " +
HRegionServer.class.getSimpleName());
VersionInfo.logVersion();
Configuration conf = HBaseConfiguration.create();
CommandLine commandLine = null;
+
+Configuration commandLineConf = new Configuration();
Review comment:
Should this be HBaseConfiguration.create to pick up hbase-site?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Create assembly and scripts to start Kafka Proxy
>
>
> Key: HBASE-21002
> URL: https://issues.apache.org/jira/browse/HBASE-21002
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
>Reporter: Mike Wingert
>Assignee: Mike Wingert
>Priority: Minor
>
> Add scripts for running and assembly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[
https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620115#comment-16620115
]
ASF GitHub Bot commented on HBASE-21002:
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218127341
##
File path: kafka/hbase-kafka-proxy/pom.xml
##
@@ -76,43 +77,40 @@
org.apache.avro
avro
-
+
org.apache.hbase.connectors
hbase-kafka-model
-
- org.apache.hbase
- hbase-common
-
+
org.apache.hbase
hbase-common
test-jar
test
-
- org.apache.hbase
- hbase-client
-
-
- org.apache.hbase
- hbase-zookeeper
-
+
org.apache.hbase
hbase-server
+ provided
org.apache.hbase
hbase-annotations
+ compile
org.apache.hbase
hbase-annotations
test-jar
test
+
+ org.apache.yetus
+ audience-annotations
+ ${audience-annotations.version}
+
org.apache.kafka
kafka-clients
Review comment:
Can this rely on the shaded hbase client? Or it don't matter because it is
"provided"?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Create assembly and scripts to start Kafka Proxy
>
>
> Key: HBASE-21002
> URL: https://issues.apache.org/jira/browse/HBASE-21002
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
>Reporter: Mike Wingert
>Assignee: Mike Wingert
>Priority: Minor
>
> Add scripts for running and assembly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[
https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620111#comment-16620111
]
ASF GitHub Bot commented on HBASE-21002:
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218126550
##
File path: kafka/README
##
@@ -0,0 +1,126 @@
+Hbase Kafka Proxy
+
+Welcome to the hbase kafka proxy. The purpose of this proxy is to act as a
'fake peer'. It
+receives replication events from it's peer and applies a set of rules (stored
in
+kafka-route-rules.xml) to determine if the event should be forwared to a
topic. If the
+mutation matches one of the rules, the mutation is converted to an avro object
and the
+item is placed into the topic.
+
+The service sets up a bare bones region server, so it will use the values in
hbase-site.xml. If
+you wish to override those values, pass them as -Dkey=value.
+
+To Use:
+
+1. Make sure the hbase command is in your path. The proxy uses the 'hbase
classpath' command to
+find the hbase libraries.
+
+2. Create any topics in your kafka broker that you wish to use.
+
+3. set up kafka-route-rules.xml. This file controls how the mutations are
routed. There are
+two kinds of rules: route and drop. drop: any mutation that matches this rule
will be dropped.
+route: any mutation that matches this rule will be routed to the configured
topic.
+
+Each rule has the following parameters:
+- table
+- columnFamily
+- qualifier
+
+The qualifier parameter can contain simple wildcard expressions (start and end
only).
+
+Examples
+
+
+
+
+
+
+This causes all mutations done to default:mytable to be routed to kafka topic
'foo'
+
+
+
+
+
+
+This will cause all mutations from default:mytable columnFamily mycf and
qualifier myqualifier
+to be routed to mykafkatopic.
+
+
+
+
+
+
+
+This combination will route all mutations from default:mytable columnFamily
mycf to mykafkatopic
+unless they start with 'secret'. Items matching that rule will be dropped.
The way the rule is
+written, all other mutations for column family mycf will be routed to the
'mykafka' topic.
+
+4. Service arguments
+
+--kafkabrokers (or -b)
+--routerulesfile (or -r)
+--kafkaproperties (or -f)
+--peername (or -p) name of hbase peer to use (defaults to hbasekafka)
+--znode (or -z) root znode (defaults to /kafkaproxy)
+--enablepeer (or -e) enable peer on startup (defaults to false)]
+--auto (or -a) auto create peer
+
+
+5. start the service.
+ - make sure the hbase command is in your path
+ - ny default, the service looks for route-rules.xml in the conf directory,
you can specify a
+ differeent file or location with the -r argument
+
+bin/hbase-connectors-daemon.sh start kafkaproxy -a -e -p wootman -b
localhost:9092 -r ~/kafka-route-rules.xml
+
+this:
+- starts the kafka proxy
+- passes the -a. The proxy will create the replication peer specified by -p
if it does not exist
+ (not required, but it savecs some busy work).
+- enables the peer (-e) the proxy will enable the peer when the service starts
(not required, you can
+ manually enable the peer in the hbase shell)
+
+
+Notes:
+1. The proxy will connect to the zookeeper in hbase-site.xml by default. You
can override this by
+ passing -Dhbase.zookeeper.quorum
+
+ bin/hbase-connectors-daemon.sh start kafkaproxy
-Dhbase.zookeeper.quorum=localhost:1234 . other args
+
+2. route rules only support unicode characters.
+3. I do not have access to a secured hadoop clsuter to test this on.
+
+Message format
+
+Messages are in avro format, this is the schema:
+
+{"namespace": "org.apache.hadoop.hbase.kafka",
+ "type": "record",
+ "name": "HBaseKafkaEvent",
+ "fields": [
+{"name": "key", "type": "bytes"},
+{"name": "timestamp", "type": "long" },
+{"name": "delete", "type": "boolean" },
+{"name": "value", "type": "bytes"},
+{"name": "qualifier", "type": "bytes"},
+{"name": "family", "type": "bytes"},
+{"name": "table", "type": "bytes"}
+ ]
+}
+
+Any language that supports Avro should be able to consume the messages off the
topic.
+
+
+Testing Utility
+
+A utility is included to test the routing rules.
+
+bin/hbase-connectors-daemon.sh start kafkaproxytest -k -t
+
+the messages will be dumped in string format under logs/
+
+TODO:
+1. Some properties passed into the region server are hard-coded.
+2. The avro objects should be generic
+3. Allow rules to be refreshed without a restart
+4. Get this tested on a secure (TLS & Kerberos) enabled cluster.
Review comment:
this is great.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For q
[GitHub] saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218127341
##
File path: kafka/hbase-kafka-proxy/pom.xml
##
@@ -76,43 +77,40 @@
org.apache.avro
avro
-
+
org.apache.hbase.connectors
hbase-kafka-model
-
- org.apache.hbase
- hbase-common
-
+
org.apache.hbase
hbase-common
test-jar
test
-
- org.apache.hbase
- hbase-client
-
-
- org.apache.hbase
- hbase-zookeeper
-
+
org.apache.hbase
hbase-server
+ provided
org.apache.hbase
hbase-annotations
+ compile
org.apache.hbase
hbase-annotations
test-jar
test
+
+ org.apache.yetus
+ audience-annotations
+ ${audience-annotations.version}
+
org.apache.kafka
kafka-clients
Review comment:
Can this rely on the shaded hbase client? Or it don't matter because it is
"provided"?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors
saintstack commented on a change in pull request #3: HBASE-21002 make an
assembly for hbase-connectors
URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218127793
##
File path:
kafka/hbase-kafka-proxy/src/main/java/org/apache/hadoop/hbase/kafka/KafkaProxy.java
##
@@ -154,19 +150,28 @@ public static void main(String[] args) throws Exception {
options.addOption("e", "enablepeer", false,
"enable peer on startup (defaults to false)");
-
LOG.info("STARTING executorService " +
HRegionServer.class.getSimpleName());
VersionInfo.logVersion();
Configuration conf = HBaseConfiguration.create();
CommandLine commandLine = null;
+
+Configuration commandLineConf = new Configuration();
Review comment:
Should this be HBaseConfiguration.create to pick up hbase-site?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[ https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620113#comment-16620113 ] ASF GitHub Bot commented on HBASE-21002: saintstack commented on a change in pull request #3: HBASE-21002 make an assembly for hbase-connectors URL: https://github.com/apache/hbase-connectors/pull/3#discussion_r218108464 ## File path: kafka/README ## @@ -0,0 +1,126 @@ +Hbase Kafka Proxy Review comment: Should this be in the README.md? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create assembly and scripts to start Kafka Proxy > > > Key: HBASE-21002 > URL: https://issues.apache.org/jira/browse/HBASE-21002 > Project: HBase > Issue Type: Sub-task > Components: hbase-connectors >Reporter: Mike Wingert >Assignee: Mike Wingert >Priority: Minor > > Add scripts for running and assembly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620116#comment-16620116 ] Ted Yu commented on HBASE-21102: I will commit tomorrow. The failing test is in hbase-rsgroups, a different module. So qa didn't run that test. > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0, 2.2.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: 21102.addendum2.txt, HBASE-21102_1.patch, > HBASE-21102_2.patch, HBASE-21102_3.patch, HBASE-21102_4.patch, > HBASE-21102_addendum.patch, HBASE-21102_addendum.patch, > HBASE-21102_addendum.patch, HBASE-21102_branch-2.1.patch, > HBASE-21102_branch-2.1.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21206) Scan with batch size may return incomplete cells
[
https://issues.apache.org/jira/browse/HBASE-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620122#comment-16620122
]
Hadoop QA commented on HBASE-21206:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 85
unchanged - 1 fixed = 85 total (was 86) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m
8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}176m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21206 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940337/HBASE-21206.v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f9181bc15e31 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cebb725a9f |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14450/testReport/ |
| Max. process+thread count | 4901 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14450/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
[jira] [Commented] (HBASE-21002) Create assembly and scripts to start Kafka Proxy
[ https://issues.apache.org/jira/browse/HBASE-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620123#comment-16620123 ] stack commented on HBASE-21002: --- [~MikeWingert] And to answer you question I think what you have posted looks great. There's a few small questions in the above and then lets get it committed. Its a great start. We can file follow-ons to do cleanup or reorg once others start taking a look (e.g. Could you make use of the hbase-shaded-client instead of depend directly on hbase-client and hbase-server? It looks like you could because you don't use HBaseTestingUtility... If you did, it would make it harder. Can do in a follow-on). > Create assembly and scripts to start Kafka Proxy > > > Key: HBASE-21002 > URL: https://issues.apache.org/jira/browse/HBASE-21002 > Project: HBase > Issue Type: Sub-task > Components: hbase-connectors >Reporter: Mike Wingert >Assignee: Mike Wingert >Priority: Minor > > Add scripts for running and assembly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21200) Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner.
[ https://issues.apache.org/jira/browse/HBASE-21200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620124#comment-16620124 ] Toshihiro Suzuki commented on HBASE-21200: -- ping [~dongjin2193]. > Memstore flush doesn't finish because of seekToPreviousRow() in memstore > scanner. > - > > Key: HBASE-21200 > URL: https://issues.apache.org/jira/browse/HBASE-21200 > Project: HBase > Issue Type: Bug > Components: Scanners >Reporter: dongjin2193.jeon >Priority: Major > > The issue of delaying memstore flush still occurs after backport hbase-15871. > Reverse scan takes a long time to seek previous row in the memstore full of > deleted cells. > > jstack : > "MemStoreFlusher.0" #114 prio=5 os_prio=0 tid=0x7fa3d0729000 nid=0x486a > waiting on condition [0x7fa3b9b6b000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0xa465fe60> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) > at > org.apache.hadoop.hbase.regionserver.*StoreScanner.updateReaders(StoreScanner.java:695)* > at > org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1127) > at > org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1106) > at > org.apache.hadoop.hbase.regionserver.HStore.access$600(HStore.java:130) > at > org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2455) > at > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2519) > at > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2256) > at > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2218) > at > org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2110) > at > org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2036) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:501) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:75) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259) > at java.lang.Thread.run(Thread.java:748) > > "RpcServer.FifoWFPBQ.default.handler=27,queue=0,port=16020" #65 daemon prio=5 > os_prio=0 tid=0x7fa3e628 nid=0x4801 runnable [0x7fa3bd29a000] > java.lang.Thread.State: RUNNABLE > at > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner.getNext(DefaultMemStore.java:780) > at > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner.seekInSubLists(DefaultMemStore.java:826) > - locked <0xb45aa5b8> (a > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner) > at > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner.seek(DefaultMemStore.java:818) > - locked <0xb45aa5b8> (a > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner) > at > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner.seekToPreviousRow(DefaultMemStore.java:1000) > - locked <0xb45aa5b8> (a > org.apache.hadoop.hbase.regionserver.DefaultMemStore$MemStoreScanner) > at > org.apache.hadoop.hbase.regionserver.ReversedKeyValueHeap.next(ReversedKeyValueHeap.java:136) > at > org.apache.hadoop.hbase.regionserver.*StoreScanner.next(StoreScanner.java:629)* > at > org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion
