[jira] [Updated] (HBASE-21660) Apply the cell to right memstore for increment/append operation
[ https://issues.apache.org/jira/browse/HBASE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21660: --- Description: HBASE-21643 introduced two new coprocessor methods postIncrementBeforeWAL and postAppendBeforeWAL instead of the old coprocessor postMutationsBeforeWAL. These coprocessor methods give coprocessors to update the new cells before apply to WAL or memstore. But the cell's column family may be changed, too. So it should apply the new cell to memstore depend on the cell's column family. > Apply the cell to right memstore for increment/append operation > --- > > Key: HBASE-21660 > URL: https://issues.apache.org/jira/browse/HBASE-21660 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > HBASE-21643 introduced two new coprocessor methods postIncrementBeforeWAL and > postAppendBeforeWAL instead of the old coprocessor postMutationsBeforeWAL. > These coprocessor methods give coprocessors to update the new cells before > apply to WAL or memstore. But the cell's column family may be changed, too. > So it should apply the new cell to memstore depend on the cell's column > family. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21660) Apply the cell to right memstore for increment/append operation
Guanghao Zhang created HBASE-21660: -- Summary: Apply the cell to right memstore for increment/append operation Key: HBASE-21660 URL: https://issues.apache.org/jira/browse/HBASE-21660 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang Assignee: Guanghao Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[
https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730582#comment-16730582
]
Hadoop QA commented on HBASE-21659:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}127m
56s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21659 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12953279/HBASE-21659.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 54b25eb9860c 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7755d4beed |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15411/testReport/ |
| Max. process+thread count | 4845 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15411/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Avoid to load
[jira] [Updated] (HBASE-21588) Procedure v2 wal splitting implementation
[ https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21588: - Attachment: HBASE-21588.master.016.patch > Procedure v2 wal splitting implementation > - > > Key: HBASE-21588 > URL: https://issues.apache.org/jira/browse/HBASE-21588 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21588.master.003.patch, > HBASE-21588.master.004.patch, HBASE-21588.master.005.patch, > HBASE-21588.master.006.patch, HBASE-21588.master.007.patch, > HBASE-21588.master.008.patch, HBASE-21588.master.009.patch, > HBASE-21588.master.010.patch, HBASE-21588.master.011.patch, > HBASE-21588.master.012.patch, HBASE-21588.master.013.patch, > HBASE-21588.master.014.patch, HBASE-21588.master.015.patch, > HBASE-21588.master.016.patch > > > create a sub task to submit the implementation of procedure v2 wal splitting -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21588) Procedure v2 wal splitting implementation
[
https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730571#comment-16730571
]
Hadoop QA commented on HBASE-21588:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} The patch passed checkstyle in hbase-common {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} hbase-server: The patch generated 0 new + 271
unchanged - 1 fixed = 271 total (was 272) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 31s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
45s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}126m
6s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 34s{color} |
{color:black} {color}
[jira] [Commented] (HBASE-21646) Flakey TestTableSnapshotInputFormat; DisableTable not completing...
[
https://issues.apache.org/jira/browse/HBASE-21646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730559#comment-16730559
]
Hudson commented on HBASE-21646:
Results for branch branch-2.1
[build #722 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/722/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/722//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/722//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/722//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Flakey TestTableSnapshotInputFormat; DisableTable not completing...
> ---
>
> Key: HBASE-21646
> URL: https://issues.apache.org/jira/browse/HBASE-21646
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.3
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.2, 2.0.4
>
> Attachments: HBASE-21646.branch-2.0.001.patch
>
>
> Fails about 50% of time in the GCE builds...
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.0/lastSuccessfulBuild/artifact/dashboard.html
> We are disabling table to delete at end of test when multi regions and we
> don't complete... pid=21:
> {code}
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> assignment.RegionTransitionProcedure(297): Received report CLOSED seqId=-1,
> pid=27, ppid=21, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> procedure.MasterProcedureScheduler(339): Add TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=1) to run queue because: pid=27, ppid=21,
> state=RUNNABLE: REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135 has lock
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=55246]
> assignment.RegionTransitionProcedure(297): Received report CLOSED seqId=-1,
> pid=26, ppid=21, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG [PEWorker-1]
> procedure.MasterProcedureScheduler(349): Remove TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=0) from run queue because: queue is empty after
> polling out pid=27, ppid=21, state=RUNNABLE:REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG [PEWorker-1]
> assignment.RegionTransitionProcedure(394): Finishing pid=27, ppid=21,
> state=RUNNABLE:REGION_TRANSITION_FINISH, locked=true; UnassignProcedure
> table=testWithMapReduce, region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51, 55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=55246]
> procedure.MasterProcedureScheduler(339): Add TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=1) to run queue because: pid=26, ppid=21,
> state=RUNNABLE: REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135 has lock
> 2018-12-26 20:52:17,180 DEBUG [PEWorker-14]
> procedure.MasterProcedureScheduler(349): Remove TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=0) from run queue because: queue is empty after
> polling out pid=26, ppid=21, state=RUNNABLE:REGION_TRANSITION_FINISH,
> locked=true;
[jira] [Updated] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21659: --- Status: Patch Available (was: Open) > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21659.master.001.patch, > HBASE-21659.master.002.patch > > > When there are same coprocessor in system coprocessor config and table > descriptor. It will load the coprocessor twice and the coprocessor method may > run twice, too. It should avoid to load duplicate coprocessors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21659: --- Attachment: HBASE-21659.master.002.patch > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21659.master.001.patch, > HBASE-21659.master.002.patch > > > When there are same coprocessor in system coprocessor config and table > descriptor. It will load the coprocessor twice and the coprocessor method may > run twice, too. It should avoid to load duplicate coprocessors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21659: --- Description: When there are same coprocessor in system coprocessor config and table descriptor. It will load the coprocessor twice and the coprocessor method may run twice, too. This should be avoid. (was: When there are same coprocessor in system coprocessor config and table descriptor. It will load the coprocessor twice and the coprocessor method may run twice, too.) > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21659.master.001.patch > > > When there are same coprocessor in system coprocessor config and table > descriptor. It will load the coprocessor twice and the coprocessor method may > run twice, too. This should be avoid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21659: --- Description: When there are same coprocessor in system coprocessor config and table descriptor. It will load the coprocessor twice and the coprocessor method may run twice, too. It should avoid to load duplicate coprocessors. (was: When there are same coprocessor in system coprocessor config and table descriptor. It will load the coprocessor twice and the coprocessor method may run twice, too. This should be avoid.) > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21659.master.001.patch > > > When there are same coprocessor in system coprocessor config and table > descriptor. It will load the coprocessor twice and the coprocessor method may > run twice, too. It should avoid to load duplicate coprocessors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21646) Flakey TestTableSnapshotInputFormat; DisableTable not completing...
[
https://issues.apache.org/jira/browse/HBASE-21646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730550#comment-16730550
]
Hudson commented on HBASE-21646:
Results for branch branch-2.0
[build #1207 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Flakey TestTableSnapshotInputFormat; DisableTable not completing...
> ---
>
> Key: HBASE-21646
> URL: https://issues.apache.org/jira/browse/HBASE-21646
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.3
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.2, 2.0.4
>
> Attachments: HBASE-21646.branch-2.0.001.patch
>
>
> Fails about 50% of time in the GCE builds...
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.0/lastSuccessfulBuild/artifact/dashboard.html
> We are disabling table to delete at end of test when multi regions and we
> don't complete... pid=21:
> {code}
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> assignment.RegionTransitionProcedure(297): Received report CLOSED seqId=-1,
> pid=27, ppid=21, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> procedure.MasterProcedureScheduler(339): Add TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=1) to run queue because: pid=27, ppid=21,
> state=RUNNABLE: REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135 has lock
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=55246]
> assignment.RegionTransitionProcedure(297): Received report CLOSED seqId=-1,
> pid=26, ppid=21, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG [PEWorker-1]
> procedure.MasterProcedureScheduler(349): Remove TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=0) from run queue because: queue is empty after
> polling out pid=27, ppid=21, state=RUNNABLE:REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG [PEWorker-1]
> assignment.RegionTransitionProcedure(394): Finishing pid=27, ppid=21,
> state=RUNNABLE:REGION_TRANSITION_FINISH, locked=true; UnassignProcedure
> table=testWithMapReduce, region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51, 55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=55246]
> procedure.MasterProcedureScheduler(339): Add TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=1) to run queue because: pid=26, ppid=21,
> state=RUNNABLE: REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135 has lock
> 2018-12-26 20:52:17,180 DEBUG [PEWorker-14]
> procedure.MasterProcedureScheduler(349): Remove TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=0) from run queue because: queue is empty after
> polling out pid=26, ppid=21, state=RUNNABLE:REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
>
[jira] [Commented] (HBASE-21555) Create 2.0.4 release
[
https://issues.apache.org/jira/browse/HBASE-21555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730551#comment-16730551
]
Hudson commented on HBASE-21555:
Results for branch branch-2.0
[build #1207 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1207//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Create 2.0.4 release
>
>
> Key: HBASE-21555
> URL: https://issues.apache.org/jira/browse/HBASE-21555
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments: Screen Shot 2018-12-05 at 8.38.32 PM.png
>
>
> Roll new 2.1 because of memory leak. See HBASE-21551
> Branch-2.0 was doing nicely. 10 of the last 14 passed here is a run of 6
> back-to-back that all passed. !Screen Shot 2018-12-05 at 8.38.32 PM.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21659: --- Description: When there are same coprocessor in system coprocessor config and table descriptor. It will load the coprocessor twice and the coprocessor method may run twice, too. > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21659.master.001.patch > > > When there are same coprocessor in system coprocessor config and table > descriptor. It will load the coprocessor twice and the coprocessor method may > run twice, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21659: --- Attachment: HBASE-21659.master.001.patch > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21659.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
Guanghao Zhang created HBASE-21659: -- Summary: Avoid to load duplicate coprocessors in system config and table descriptor Key: HBASE-21659 URL: https://issues.apache.org/jira/browse/HBASE-21659 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang Assignee: Guanghao Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730547#comment-16730547
]
xuqinya commented on HBASE-21656:
-
[~maoling]，Thank you for testing.I also encountered your problem, then I
skipped the check license . You can try HBASE-21135, I tested it solved
HBase-Shaded Module, but there are still problems in Apache HBase - Shaded -
MapReduce on the Windows OS.
{code:java}
[INFO] Apache HBase - Shaded .. SUCCESS [ 1.115 s]
[INFO] Apache HBase - Shaded - Client (with Hadoop bundled) SUCCESS [ 30.302 s]
[INFO] Apache HBase - Shaded - Client . SUCCESS [ 28.375 s]
[INFO] Apache HBase - Shaded - MapReduce .. FAILURE [ 6.677 s]
[INFO] Apache HBase - External Block Cache SKIPPED
[INFO] Apache HBase - Assembly SKIPPED
[INFO] Apache HBase Shaded Packaging Invariants ... SKIPPED
[INFO] Apache HBase Shaded Packaging Invariants (with Hadoop bundled) SKIPPED
[INFO] Apache HBase - Archetypes .. SKIPPED
[INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED
[INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED
[INFO] Apache HBase - Archetype builder ... SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 08:59 min
[INFO] Finished at: 2018-12-29T11:25:09+08:00
[INFO] Final Memory: 217M/967M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M1:enforce
(check-aggregate-license) on project hbase-shaded-mapreduce: Some Enforcer
rules have failed. Look above for specific messages explaining why the rule
failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
{code}
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling updated HBASE-21656:
Status: Patch Available (was: Open)
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730535#comment-16730535
]
Hadoop QA commented on HBASE-21656:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21656 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12953223/HBASE-21656.master.0001.patch
|
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux af35b61915ca 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7755d4beed |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| shellcheck | v0.4.4 |
| Max. process+thread count | 43 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15410/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730531#comment-16730531
]
maoling edited comment on HBASE-21656 at 12/29/18 2:46 AM:
---
[~xu qinya]
I kill off the QA Report.Let us listen to her about building on the linux OS
[~nihaljain.cs] Let me apply the patch to test
was (Author: maoling):
[~xu qinya]
I kill off the QA Report.Let us listen her about building on the linux OS
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730531#comment-16730531
]
maoling commented on HBASE-21656:
-
[~xu qinya]
I kill off the QA Report.Let us listen her about building on the linux OS
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21588) Procedure v2 wal splitting implementation
[ https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21588: - Attachment: HBASE-21588.master.015.patch > Procedure v2 wal splitting implementation > - > > Key: HBASE-21588 > URL: https://issues.apache.org/jira/browse/HBASE-21588 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21588.master.003.patch, > HBASE-21588.master.004.patch, HBASE-21588.master.005.patch, > HBASE-21588.master.006.patch, HBASE-21588.master.007.patch, > HBASE-21588.master.008.patch, HBASE-21588.master.009.patch, > HBASE-21588.master.010.patch, HBASE-21588.master.011.patch, > HBASE-21588.master.012.patch, HBASE-21588.master.013.patch, > HBASE-21588.master.014.patch, HBASE-21588.master.015.patch > > > create a sub task to submit the implementation of procedure v2 wal splitting -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21588) Procedure v2 wal splitting implementation
[ https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21588: - Attachment: (was: HBASE-21588.master.015.patch) > Procedure v2 wal splitting implementation > - > > Key: HBASE-21588 > URL: https://issues.apache.org/jira/browse/HBASE-21588 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21588.master.003.patch, > HBASE-21588.master.004.patch, HBASE-21588.master.005.patch, > HBASE-21588.master.006.patch, HBASE-21588.master.007.patch, > HBASE-21588.master.008.patch, HBASE-21588.master.009.patch, > HBASE-21588.master.010.patch, HBASE-21588.master.011.patch, > HBASE-21588.master.012.patch, HBASE-21588.master.013.patch, > HBASE-21588.master.014.patch > > > create a sub task to submit the implementation of procedure v2 wal splitting -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21658) Should get the meta replica number from zk instead of config at client side
[ https://issues.apache.org/jira/browse/HBASE-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730524#comment-16730524 ] maoling commented on HBASE-21658: - [~Apache9] interested in this issue. just no activities in the New Year's Day(cry) > Should get the meta replica number from zk instead of config at client side > --- > > Key: HBASE-21658 > URL: https://issues.apache.org/jira/browse/HBASE-21658 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Duo Zhang >Assignee: maoling >Priority: Critical > > For now, the ZKAsyncRegistry will use the znodePaths.metaReplicaZNodes for > getting meta replicas. But actually, this config should only be used by > HMaster. At client side, we should list the zk directory to get all the > available meta replicas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21658) Should get the meta replica number from zk instead of config at client side
[ https://issues.apache.org/jira/browse/HBASE-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling reassigned HBASE-21658: --- Assignee: maoling > Should get the meta replica number from zk instead of config at client side > --- > > Key: HBASE-21658 > URL: https://issues.apache.org/jira/browse/HBASE-21658 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Duo Zhang >Assignee: maoling >Priority: Critical > > For now, the ZKAsyncRegistry will use the znodePaths.metaReplicaZNodes for > getting meta replicas. But actually, this config should only be used by > HMaster. At client side, we should list the zk directory to get all the > available meta replicas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21646) Flakey TestTableSnapshotInputFormat; DisableTable not completing...
[
https://issues.apache.org/jira/browse/HBASE-21646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21646:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch-2.0+
> Flakey TestTableSnapshotInputFormat; DisableTable not completing...
> ---
>
> Key: HBASE-21646
> URL: https://issues.apache.org/jira/browse/HBASE-21646
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.3
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.2, 2.0.4
>
> Attachments: HBASE-21646.branch-2.0.001.patch
>
>
> Fails about 50% of time in the GCE builds...
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.0/lastSuccessfulBuild/artifact/dashboard.html
> We are disabling table to delete at end of test when multi regions and we
> don't complete... pid=21:
> {code}
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> assignment.RegionTransitionProcedure(297): Received report CLOSED seqId=-1,
> pid=27, ppid=21, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> procedure.MasterProcedureScheduler(339): Add TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=1) to run queue because: pid=27, ppid=21,
> state=RUNNABLE: REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135 has lock
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=55246]
> assignment.RegionTransitionProcedure(297): Received report CLOSED seqId=-1,
> pid=26, ppid=21, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG [PEWorker-1]
> procedure.MasterProcedureScheduler(349): Remove TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=0) from run queue because: queue is empty after
> polling out pid=27, ppid=21, state=RUNNABLE:REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG [PEWorker-1]
> assignment.RegionTransitionProcedure(394): Finishing pid=27, ppid=21,
> state=RUNNABLE:REGION_TRANSITION_FINISH, locked=true; UnassignProcedure
> table=testWithMapReduce, region=b66d77b9dd3eaed5b4f8c89872bb56ca,
> server=192.168.1.51, 55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,179 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=55246]
> procedure.MasterProcedureScheduler(339): Add TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=1) to run queue because: pid=26, ppid=21,
> state=RUNNABLE: REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135 has lock
> 2018-12-26 20:52:17,180 DEBUG [PEWorker-14]
> procedure.MasterProcedureScheduler(349): Remove TableQueue(testWithMapReduce,
> xlock=false sharedLock=8 size=0) from run queue because: queue is empty after
> polling out pid=26, ppid=21, state=RUNNABLE:REGION_TRANSITION_FINISH,
> locked=true; UnassignProcedure table=testWithMapReduce,
> region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,180 DEBUG [PEWorker-14]
> assignment.RegionTransitionProcedure(394): Finishing pid=26, ppid=21,
> state=RUNNABLE:REGION_TRANSITION_FINISH, locked=true; UnassignProcedure
> table=testWithMapReduce, region=71231f8d1e9b8d4dc83519a6b2b0602a,
> server=192.168.1.51, 55250,1545886322135; rit=CLOSING,
> location=192.168.1.51,55250,1545886322135
> 2018-12-26 20:52:17,405 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> master.MasterRpcServices(1152): Checking to see if procedure is done pid=21
> 2018-12-26 20:52:17,911 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=55246]
> master.MasterRpcServices(1152): Checking to see if procedure is done pid=21
> 2018-12-26 20:52:18,917 DEBUG
>
[jira] [Commented] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730480#comment-16730480
]
Hadoop QA commented on HBASE-21639:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}124m
47s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21639 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12953264/HBASE-21639.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c323bea4a5d7 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b620334c20 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15408/testReport/ |
| Max. process+thread count | 5271 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15408/console |
[jira] [Commented] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730456#comment-16730456
]
Peter Somogyi commented on HBASE-21547:
---
Yes, right now PreCommit job has
EXCLUDE_TESTS_URL=[https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/]
by default. The plan is to remove the default value and use the branch
specific exclude list. If for any reason we'd like to use other exclude list we
can just set the value for a single test run.
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch,
> HBASE-21547.master.002.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
> -Dtest.exclude.pattern=**/master.cleaner.TestSnapshotFromMaster.java,**/client.TestRestoreSnapshotFromClientAfterSplittingRegions.java,**/regionserver.TestRegionMergeTransactionOnCluster.java,**/client.TestCloneSnapshotFromClientAfterSplittingRegion.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestAMAssignWithRandExec.java,**/client.TestMobCloneSnapshotFromClientAfterSplittingRegion.java,**/regionserver.TestCompactingToCellFlatMapMemStore.java,**/replication.TestReplicationSmallTestsSync.java,**/TestMultiVersions.java,**/client.TestMobRestoreSnapshotFromClientAfterSplittingRegions.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/regionserver.TestRegionServerAbortTimeout.java,**/replication.TestMasterReplication.java,**/backup.TestIncrementalBackupWithBulkLoad.java,**/master.replication.TestRegisterPeerWorkerWhenRestarting.java
> clean test -fae > /testptch/patchprocess/patch-unit-hbase-thrift.txt 2>&1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21639:
-
Attachment: HBASE-21639.001.patch
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21639:
-
Fix Version/s: 3.0.0
Status: Patch Available (was: Open)
Simple patch, kindly review.
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0, 1.3.1
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21639:
-
Summary: maxHeapUsage value not read properly from config during
EntryBuffers initialization (was: maxHeapUsage value not read properly during
EntryBuffers initialization)
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21639) maxHeapUsage value not read properly during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21639:
-
Description:
{code:java|title=WALSplitter.java|borderStyle=solid}
entryBuffers = new EntryBuffers(controller,
this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
1024),
splitWriterCreationBounded);
{code}
In above case, EntryBuffers can't support maxHeapUsage in GB size.
The parameter type of the new EntryBuffers() is long, but the conf max value is
INT.MAX.
this is wrong？it should be getLong？
was:
{code:title=WALSplitter.java|borderStyle=solid}
entryBuffers = new EntryBuffers(controller,
this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
1024),
splitWriterCreationBounded);
{code}
the parameter type of the new EntryBuffers() is long, but the conf max value is
INT.MAX
this is wrong？it should be getLong？
> maxHeapUsage value not read properly during EntryBuffers initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21639) maxHeapUsage value not read properly during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21639:
-
Summary: maxHeapUsage value not read properly during EntryBuffers
initialization (was: hbase.regionserver.hlog.splitlog.buffersize does not
support GB size)
> maxHeapUsage value not read properly during EntryBuffers initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
>
> {code:title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> the parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21639) hbase.regionserver.hlog.splitlog.buffersize does not support GB size
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar reassigned HBASE-21639:
Assignee: Pankaj Kumar
> hbase.regionserver.hlog.splitlog.buffersize does not support GB size
>
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
>
> {code:title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> the parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21649) Complete Thrift2
[ https://issues.apache.org/jira/browse/HBASE-21649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730436#comment-16730436 ] Sean Busbey commented on HBASE-21649: - [~allan163] sounds good. > Complete Thrift2 > > > Key: HBASE-21649 > URL: https://issues.apache.org/jira/browse/HBASE-21649 > Project: HBase > Issue Type: Umbrella >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Thrift1 and Thrift2 coexists in our project for a very long time. > Functionality is more complete in thrift1 but its interface design is bad for > adding new features（so we have get(), getVer()，getVerTs,getRowWithColumns() > and so many other methods for a single get request, this is bad). Thrift2 has > a more clean interface and structure definition, making our user more easy to > use. But, it has not been updated for a long time, lacking of DDL method is a > major weakness. > I think we should complete Thrift2 and supersede Thrift1, making Thrift2 as > the standard multi language definition. This is a umbrella issue to make it > happen. > The plan would be: > 1. Complete the DDL interface of thrift2 > 2. Making thrift2 server be able to handle thrift1 requests, user don't have > to choose which thrift server they need to start > 3. Deprecate thrift1(need to discuss and only go to 3.x) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730434#comment-16730434
]
Sean Busbey commented on HBASE-21547:
-
so the idea is on our precommit job we'd stop defining EXCLUDE_TESTS_URL, hit
this logic, and thus get the correct branch.
+1 looks good.
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch,
> HBASE-21547.master.002.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
> -Dtest.exclude.pattern=**/master.cleaner.TestSnapshotFromMaster.java,**/client.TestRestoreSnapshotFromClientAfterSplittingRegions.java,**/regionserver.TestRegionMergeTransactionOnCluster.java,**/client.TestCloneSnapshotFromClientAfterSplittingRegion.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestAMAssignWithRandExec.java,**/client.TestMobCloneSnapshotFromClientAfterSplittingRegion.java,**/regionserver.TestCompactingToCellFlatMapMemStore.java,**/replication.TestReplicationSmallTestsSync.java,**/TestMultiVersions.java,**/client.TestMobRestoreSnapshotFromClientAfterSplittingRegions.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/regionserver.TestRegionServerAbortTimeout.java,**/replication.TestMasterReplication.java,**/backup.TestIncrementalBackupWithBulkLoad.java,**/master.replication.TestRegisterPeerWorkerWhenRestarting.java
> clean test -fae > /testptch/patchprocess/patch-unit-hbase-thrift.txt 2>&1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-9888) HBase replicates edits written before the replication peer is created
[ https://issues.apache.org/jira/browse/HBASE-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730425#comment-16730425 ] Pankaj Kumar commented on HBASE-9888: - We can make WALKeyWriteTimeBasedFilter internal like SystemTableWALEntryFilter, no need to make it configurable. [~stack] sir, please provide your opinion. > HBase replicates edits written before the replication peer is created > - > > Key: HBASE-9888 > URL: https://issues.apache.org/jira/browse/HBASE-9888 > Project: HBase > Issue Type: Bug >Affects Versions: 1.5.0 >Reporter: Dave Latham >Assignee: Pankaj Kumar >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-9888.002.patch, HBASE-9888.branch-1.001.patch, > HBASE-9888.branch-1.002.patch, HBASE-9888.branch-2.002.patch, > HBASE-9888.branch-2.patch, HBASE-9888.branch-2.patch, HBASE-9888.patch > > > When creating a new replication peer the ReplicationSourceManager enqueues > the currently open HLog to the ReplicationSource to ship to the destination > cluster. The ReplicationSource starts at the beginning of the HLog and ships > over any pre-existing writes. > A workaround is to roll all the HLogs before enabling replication. > A little background for how it affected us - we were migrating one cluster in > a master-master pair. I.e. transitioning from A <\-> B to B <-> C. After > shutting down writes from A -> B we enabled writes from C -> B. However, > this replicated some earlier writes that were in C's HLogs that had > originated in A. Since we were running a version of HBase before HBASE-7709 > those writes then got caught in a infinite replication cycle and bringing > down region servers OOM because of HBASE-9865. > However, in general, if one wants to manage what data gets replicated, one > wouldn't expect that potentially very old writes would be included when > setting up a new replication link. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21588) Procedure v2 wal splitting implementation
[
https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730395#comment-16730395
]
Hadoop QA commented on HBASE-21588:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 1 new + 271
unchanged - 1 fixed = 272 total (was 272) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 50s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}124m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21588 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12953238/HBASE-21588.master.015.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars
[jira] [Commented] (HBASE-21159) Add shell command to switch throttle on or off
[
https://issues.apache.org/jira/browse/HBASE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730379#comment-16730379
]
Hadoop QA commented on HBASE-21159:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} hbase-client: The patch generated 1 new + 339
unchanged - 0 fixed = 340 total (was 339) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 7 new + 227
unchanged - 0 fixed = 234 total (was 227) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 21 new + 149 unchanged - 2 fixed =
170 total (was 151) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 9s{color} | {color:orange} The patch generated 14 new + 270 unchanged - 0
fixed = 284 total (was 270) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 30s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
8s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730346#comment-16730346
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #30 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/30/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/30//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/30//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/30//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21643) Introduce two new region coprocessor method and deprecated postMutationBeforeWAL
[
https://issues.apache.org/jira/browse/HBASE-21643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730328#comment-16730328
]
Hudson commented on HBASE-21643:
Results for branch master
[build #685 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/685/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce two new region coprocessor method and deprecated
> postMutationBeforeWAL
>
>
> Key: HBASE-21643
> URL: https://issues.apache.org/jira/browse/HBASE-21643
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21643.master.001.patch,
> HBASE-21643.master.002.patch, HBASE-21643.master.003.patch
>
>
> The old method postMutationBeforeWAL is not accurate about what it do. It is
> only called during increment and append. But the name is "Mutation"... And
> the javadoc only said it will be called by increment...
> {code:java}
> * Called after a new cell has been created during an increment operation, but
> before
> * it is committed to the WAL or memstore.
> {code}
>
> We use this coprocessor in our use case. And need add some cells to apply to
> WAL. So I introduced two new method postIncrementBeforeWAL and
> postAppendBeforeWAL to instead of this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21650) Add DDL operation and some other miscellaneous to thrift2
[
https://issues.apache.org/jira/browse/HBASE-21650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730329#comment-16730329
]
Hudson commented on HBASE-21650:
Results for branch master
[build #685 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/685/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add DDL operation and some other miscellaneous to thrift2
> -
>
> Key: HBASE-21650
> URL: https://issues.apache.org/jira/browse/HBASE-21650
> Project: HBase
> Issue Type: Sub-task
> Components: Thrift
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21650.patch, HBASE-21650.v2.patch,
> HBASE-21650.v3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21642) CopyTable by reading snapshot and bulkloading will save a lot of time.
[
https://issues.apache.org/jira/browse/HBASE-21642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730327#comment-16730327
]
Hudson commented on HBASE-21642:
Results for branch master
[build #685 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/685/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/685//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CopyTable by reading snapshot and bulkloading will save a lot of time.
> --
>
> Key: HBASE-21642
> URL: https://issues.apache.org/jira/browse/HBASE-21642
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21642.v1.patch
>
>
> In our HBase clusters, some users has the need to merge two diff table's
> data into one. Currently , the CopyTable will scan the source table , and
> put mutations into destination table.
> Although CopyTable with bulkload can speed a lot (compared to CopyTable with
> scan and put), it still take lots of time to scan the source table. and the
> worst thing is: CopyTable with scan table will impact the cluster's
> availablity, it cost lots of resource in RS to scanning, the cpu, memory,
> gc stw, rs handlers, disk io, network io ... etc. All those things will
> affect the availablity.
> So in our clusters, we tried to do all scanning job by using scan snapshot
> instead of scan table. it at least isolate the cpu & memory & gc resource
> between the online RS and scanning job. What's more, the snapshot scanning
> is much faster than scaning RS, and it's more stable.
> So, here, I'll make the copy table tool support snapshot scanning.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730314#comment-16730314
]
Nihal Jain commented on HBASE-21656:
Hi [~maoling], you might want to try this:
https://issues.apache.org/jira/projects/HBASE/issues/HBASE-21135
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21588) Procedure v2 wal splitting implementation
[
https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730309#comment-16730309
]
Hadoop QA commented on HBASE-21588:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} hbase-server: The patch generated 1 new + 271 unchanged
- 1 fixed = 272 total (was 272) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 44s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}143m 52s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
58s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}203m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.filter.TestFilterListOrOperatorWithBlkCnt |
| | hadoop.hbase.TestHBaseOnOtherDfsCluster |
| | hadoop.hbase.replication.TestReplicationSmallTests |
| | hadoop.hbase.TestRegionLoad |
| | hadoop.hbase.tool.TestSecureLoadIncrementalHFiles |
| | hadoop.hbase.TestMetaTableAccessor
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730300#comment-16730300
]
maoling commented on HBASE-21656:
-
yes.the hbase-common passed in the windows OS
[~xu qinya]
Do you see this build error when exec *mvn clean package -DskipTests
assembly:single* in the windows ?
{code:java}
[INFO] Reactor Summary:
[INFO]
[INFO] Apache HBase ... SUCCESS [ 36.955 s]
[INFO] Apache HBase - Checkstyle .. SUCCESS [ 7.294 s]
[INFO] Apache HBase - Build Support ... SUCCESS [ 0.436 s]
[INFO] Apache HBase - Error Prone Rules ... SUCCESS [ 8.264 s]
[INFO] Apache HBase - Annotations . SUCCESS [ 2.808 s]
[INFO] Apache HBase - Build Configuration . SUCCESS [ 3.674 s]
[INFO] Apache HBase - Shaded Protocol . SUCCESS [02:19 min]
[INFO] Apache HBase - Common .. SUCCESS [01:26 min]
[INFO] Apache HBase - Metrics API . SUCCESS [ 7.399 s]
[INFO] Apache HBase - Hadoop Compatibility SUCCESS [ 9.693 s]
[INFO] Apache HBase - Metrics Implementation .. SUCCESS [ 6.568 s]
[INFO] Apache HBase - Hadoop Two Compatibility SUCCESS [ 11.786 s]
[INFO] Apache HBase - Protocol SUCCESS [ 29.798 s]
[INFO] Apache HBase - Client .. SUCCESS [ 36.956 s]
[INFO] Apache HBase - Zookeeper ... SUCCESS [ 17.302 s]
[INFO] Apache HBase - Replication . SUCCESS [ 13.443 s]
[INFO] Apache HBase - Resource Bundle . SUCCESS [ 0.823 s]
[INFO] Apache HBase - HTTP SUCCESS [ 24.315 s]
[INFO] Apache HBase - Procedure ... SUCCESS [ 12.515 s]
[INFO] Apache HBase - Server .. SUCCESS [02:19 min]
[INFO] Apache HBase - MapReduce ... SUCCESS [ 31.879 s]
[INFO] Apache HBase - Testing Util SUCCESS [ 15.689 s]
[INFO] Apache HBase - Thrift .. SUCCESS [ 41.614 s]
[INFO] Apache HBase - RSGroup . SUCCESS [ 15.653 s]
[INFO] Apache HBase - Shell ... SUCCESS [ 12.539 s]
[INFO] Apache HBase - Coprocessor Endpoint SUCCESS [ 23.944 s]
[INFO] Apache HBase - Backup .. SUCCESS [ 15.553 s]
[INFO] Apache HBase - Integration Tests ... SUCCESS [ 24.670 s]
[INFO] Apache HBase - Rest SUCCESS [ 23.449 s]
[INFO] Apache HBase - Examples SUCCESS [ 16.970 s]
[INFO] Apache HBase - Shaded .. FAILURE [ 0.545 s]
[INFO] Apache HBase - Shaded - Client (with Hadoop bundled) SKIPPED
[INFO] Apache HBase - Shaded - Client . SKIPPED
[INFO] Apache HBase - Shaded - MapReduce .. SKIPPED
[INFO] Apache HBase - External Block Cache SKIPPED
[INFO] Apache HBase - Assembly SKIPPED
[INFO] Apache HBase Shaded Packaging Invariants ... SKIPPED
[INFO] Apache HBase Shaded Packaging Invariants (with Hadoop bundled) SKIPPED
[INFO] Apache HBase - Archetypes .. SKIPPED
[INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED
[INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED
[INFO] Apache HBase - Archetype builder ... SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 14:07 min
[INFO] Finished at: 2018-12-28T22:27:52+08:00
[INFO] Final Memory: 221M/863M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce (check-aggr
egate-license) on project hbase-shaded: Some Enforcer rules have failed. Look
above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-shaded
{code}
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL:
[jira] [Commented] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730297#comment-16730297
]
Hadoop QA commented on HBASE-21547:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21547 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12953240/HBASE-21547.master.002.patch
|
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux 007355a84c1f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b620334c20 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| shellcheck | v0.4.4 |
| Max. process+thread count | 52 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15407/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch,
> HBASE-21547.master.002.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
>
[jira] [Commented] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730296#comment-16730296
]
Hadoop QA commented on HBASE-21547:
---
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://builds.apache.org/job/PreCommit-HBASE-Build/15407/console in case of
problems.
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch,
> HBASE-21547.master.002.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
> -Dtest.exclude.pattern=**/master.cleaner.TestSnapshotFromMaster.java,**/client.TestRestoreSnapshotFromClientAfterSplittingRegions.java,**/regionserver.TestRegionMergeTransactionOnCluster.java,**/client.TestCloneSnapshotFromClientAfterSplittingRegion.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestAMAssignWithRandExec.java,**/client.TestMobCloneSnapshotFromClientAfterSplittingRegion.java,**/regionserver.TestCompactingToCellFlatMapMemStore.java,**/replication.TestReplicationSmallTestsSync.java,**/TestMultiVersions.java,**/client.TestMobRestoreSnapshotFromClientAfterSplittingRegions.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/regionserver.TestRegionServerAbortTimeout.java,**/replication.TestMasterReplication.java,**/backup.TestIncrementalBackupWithBulkLoad.java,**/master.replication.TestRegisterPeerWorkerWhenRestarting.java
> clean test -fae > /testptch/patchprocess/patch-unit-hbase-thrift.txt 2>&1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21547:
--
Status: Patch Available (was: Open)
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch,
> HBASE-21547.master.002.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
> -Dtest.exclude.pattern=**/master.cleaner.TestSnapshotFromMaster.java,**/client.TestRestoreSnapshotFromClientAfterSplittingRegions.java,**/regionserver.TestRegionMergeTransactionOnCluster.java,**/client.TestCloneSnapshotFromClientAfterSplittingRegion.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestAMAssignWithRandExec.java,**/client.TestMobCloneSnapshotFromClientAfterSplittingRegion.java,**/regionserver.TestCompactingToCellFlatMapMemStore.java,**/replication.TestReplicationSmallTestsSync.java,**/TestMultiVersions.java,**/client.TestMobRestoreSnapshotFromClientAfterSplittingRegions.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/regionserver.TestRegionServerAbortTimeout.java,**/replication.TestMasterReplication.java,**/backup.TestIncrementalBackupWithBulkLoad.java,**/master.replication.TestRegisterPeerWorkerWhenRestarting.java
> clean test -fae > /testptch/patchprocess/patch-unit-hbase-thrift.txt 2>&1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21547:
--
Attachment: HBASE-21547.master.002.patch
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch,
> HBASE-21547.master.002.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
> -Dtest.exclude.pattern=**/master.cleaner.TestSnapshotFromMaster.java,**/client.TestRestoreSnapshotFromClientAfterSplittingRegions.java,**/regionserver.TestRegionMergeTransactionOnCluster.java,**/client.TestCloneSnapshotFromClientAfterSplittingRegion.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestAMAssignWithRandExec.java,**/client.TestMobCloneSnapshotFromClientAfterSplittingRegion.java,**/regionserver.TestCompactingToCellFlatMapMemStore.java,**/replication.TestReplicationSmallTestsSync.java,**/TestMultiVersions.java,**/client.TestMobRestoreSnapshotFromClientAfterSplittingRegions.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/regionserver.TestRegionServerAbortTimeout.java,**/replication.TestMasterReplication.java,**/backup.TestIncrementalBackupWithBulkLoad.java,**/master.replication.TestRegisterPeerWorkerWhenRestarting.java
> clean test -fae > /testptch/patchprocess/patch-unit-hbase-thrift.txt 2>&1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21547) Precommit uses master flaky list for other branches
[
https://issues.apache.org/jira/browse/HBASE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21547:
--
Attachment: HBASE-21547.master.001.patch
> Precommit uses master flaky list for other branches
> ---
>
> Key: HBASE-21547
> URL: https://issues.apache.org/jira/browse/HBASE-21547
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21547.master.001.patch
>
>
> Precommit job downloads the flaky exclude list for master branch when the
> uploaded patch file is made for different branches.
> As an example check
> [https://builds.apache.org/job/PreCommit-HBASE-Build/15192] which was against
> branch-1 but the unit test downloaded master's flaky list.
> {noformat}
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: Personality: patch unit
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]:
> EXCLUDE_TESTS_URL=https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 [Tue Dec 4 14:26:04 UTC 2018 INFO]: INCLUDE_TESTS_URL=
> 15:26:05 --2018-12-04 14:26:04--
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/excludes/
> 15:26:05 Resolving builds.apache.org (builds.apache.org)... 195.201.213.130,
> 2a01:4f8:c0:2cc9::2
> 15:26:05 Connecting to builds.apache.org
> (builds.apache.org)|195.201.213.130|:443... connected.
> 15:26:06 HTTP request sent, awaiting response... 200
> 15:26:06 Length: 866 [application/octet-stream]
> 15:26:06 Saving to: 'excludes'
> 15:26:06
> 15:26:06 0K 100%
> 43.0M=0s
> 15:26:06
> 15:26:06 2018-12-04 14:26:06 (43.0 MB/s) - 'excludes' saved [866/866]
> 15:26:06
> 15:26:09 cd /testptch/hbase/hbase-thrift
> 15:26:09 mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/yetus-m2/hbase-branch-1-patch-1
> -DHBasePatchProcess -Dhttps.protocols=TLSv1.2 -PrunAllTests
> -Dtest.exclude.pattern=**/master.cleaner.TestSnapshotFromMaster.java,**/client.TestRestoreSnapshotFromClientAfterSplittingRegions.java,**/regionserver.TestRegionMergeTransactionOnCluster.java,**/client.TestCloneSnapshotFromClientAfterSplittingRegion.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestAMAssignWithRandExec.java,**/client.TestMobCloneSnapshotFromClientAfterSplittingRegion.java,**/regionserver.TestCompactingToCellFlatMapMemStore.java,**/replication.TestReplicationSmallTestsSync.java,**/TestMultiVersions.java,**/client.TestMobRestoreSnapshotFromClientAfterSplittingRegions.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/regionserver.TestRegionServerAbortTimeout.java,**/replication.TestMasterReplication.java,**/backup.TestIncrementalBackupWithBulkLoad.java,**/master.replication.TestRegisterPeerWorkerWhenRestarting.java
> clean test -fae > /testptch/patchprocess/patch-unit-hbase-thrift.txt 2>&1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21588) Procedure v2 wal splitting implementation
[ https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21588: - Attachment: HBASE-21588.master.015.patch > Procedure v2 wal splitting implementation > - > > Key: HBASE-21588 > URL: https://issues.apache.org/jira/browse/HBASE-21588 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21588.master.003.patch, > HBASE-21588.master.004.patch, HBASE-21588.master.005.patch, > HBASE-21588.master.006.patch, HBASE-21588.master.007.patch, > HBASE-21588.master.008.patch, HBASE-21588.master.009.patch, > HBASE-21588.master.010.patch, HBASE-21588.master.011.patch, > HBASE-21588.master.012.patch, HBASE-21588.master.013.patch, > HBASE-21588.master.014.patch, HBASE-21588.master.015.patch > > > create a sub task to submit the implementation of procedure v2 wal splitting -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21588) Procedure v2 wal splitting implementation
[ https://issues.apache.org/jira/browse/HBASE-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21588: - Attachment: (was: HBASE-21588.master.015.patch) > Procedure v2 wal splitting implementation > - > > Key: HBASE-21588 > URL: https://issues.apache.org/jira/browse/HBASE-21588 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-21588.master.003.patch, > HBASE-21588.master.004.patch, HBASE-21588.master.005.patch, > HBASE-21588.master.006.patch, HBASE-21588.master.007.patch, > HBASE-21588.master.008.patch, HBASE-21588.master.009.patch, > HBASE-21588.master.010.patch, HBASE-21588.master.011.patch, > HBASE-21588.master.012.patch, HBASE-21588.master.013.patch, > HBASE-21588.master.014.patch > > > create a sub task to submit the implementation of procedure v2 wal splitting -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21658) Should get the meta replica number from zk instead of config at client side
Duo Zhang created HBASE-21658: - Summary: Should get the meta replica number from zk instead of config at client side Key: HBASE-21658 URL: https://issues.apache.org/jira/browse/HBASE-21658 Project: HBase Issue Type: Improvement Components: Client Reporter: Duo Zhang For now, the ZKAsyncRegistry will use the znodePaths.metaReplicaZNodes for getting meta replicas. But actually, this config should only be used by HMaster. At client side, we should list the zk directory to get all the available meta replicas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730214#comment-16730214
]
maoling edited comment on HBASE-21656 at 12/28/18 12:11 PM:
I saw this issue when I want to bulid the hbase sc in the Windows8
HBASE-18461 had not solved this issue completely.
testing your patch!
was (Author: maoling):
I saw this issue when I want to bulid the hbase sc in the Windows8
HBASE-18461 had not solved this issue completely.
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730214#comment-16730214
]
maoling commented on HBASE-21656:
-
I saw this issue when I want to bulid the hbase sc in the Windows8
HBASE-18461 had not solved this issue completely.
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Comment: was deleted (was: In HFileReaderImpl#getCell, we will new an SizeCachedKeyValue for each cell, but it only cache rowLen & keyLen now, I think we can add a serializedSize in here which may save lots of cpu.) > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730177#comment-16730177
]
xuqinya commented on HBASE-21656:
-
Aha, [~maoling]. It’s good to have money.I also use Mac, but sometimes use
Windows.
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21159) Add shell command to switch throttle on or off
[ https://issues.apache.org/jira/browse/HBASE-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21159: --- Attachment: HBASE-21159.master.003.patch > Add shell command to switch throttle on or off > -- > > Key: HBASE-21159 > URL: https://issues.apache.org/jira/browse/HBASE-21159 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21159.master.001.patch, > HBASE-21159.master.002.patch, HBASE-21159.master.003.patch > > > Add shell command to switch throttle on or off. When throttle is off, HBase > will not throttle any request. This feature may be useful in production > environment. > We can use the following commands to switch throttle: > throttle_switch true / throttle_switch false -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xuqinya updated HBASE-21656:
Attachment: HBASE-21656.master.0001.patch
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
> Attachments: HBASE-21656.master.0001.patch
>
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730168#comment-16730168
]
maoling commented on HBASE-21656:
-
[~xu qinya]
HBASE-18461 had not solved this issue completely.
But when I bought a MAC, everything is ok. it's the power of money(:D)
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730161#comment-16730161 ] Zheng Hu commented on HBASE-21657: -- In HFileReaderImpl#getCell, we will new an SizeCachedKeyValue for each cell, but it only cache rowLen & keyLen now, I think we can add a serializedSize in here which may save lots of cpu. > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Fix Version/s: 2.0.5 2.1.3 2.2.0 3.0.0 > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Description: We are evaluating the performance of branch-2, and find that the throughput of scan in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] BTW, in our XiaoMi branch, we introduce a HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells (for metric monitor), so it seems the performance loss was amplified. was: We are evaluating the performance of branch-2, and find that the throughput in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] BTW, in our XiaoMi branch, we introduce a HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells (for metric monitor), so it seems the performance loss was amplified. > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Component/s: Performance > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on > RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about > 29% cpu, Obviously, it has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Description: We are evaluating the performance of branch-2, and find that the throughput in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] BTW, in our XiaoMi branch, we introduce a HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells (for metric monitor), so it seems the performance loss was amplified, but the was: We are evaluating the performance of branch-2, and find that the throughput in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on > RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about > 29% cpu, Obviously, it has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified, but > the -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Description: We are evaluating the performance of branch-2, and find that the throughput in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] BTW, in our XiaoMi branch, we introduce a HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells (for metric monitor), so it seems the performance loss was amplified. was: We are evaluating the performance of branch-2, and find that the throughput in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] BTW, in our XiaoMi branch, we introduce a HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells (for metric monitor), so it seems the performance loss was amplified, but the > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: hbase20-ssd-100-scan-traces.svg > > > We are evaluating the performance of branch-2, and find that the throughput > in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on > RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about > 29% cpu, Obviously, it has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
Zheng Hu created HBASE-21657: Summary: PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case. Key: HBASE-21657 URL: https://issues.apache.org/jira/browse/HBASE-21657 Project: HBase Issue Type: Bug Reporter: Zheng Hu Assignee: Zheng Hu Attachments: hbase20-ssd-100-scan-traces.svg We are evaluating the performance of branch-2, and find that the throughput in SSD cluster is almost the same as HDD cluster. so I made a FlameGraph on RS, and found that the PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it has been the bottleneck in 100% scan case. See the [^hbase20-ssd-100-scan-traces.svg] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xuqinya reassigned HBASE-21656:
---
Assignee: xuqinya
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Assignee: xuqinya
>Priority: Major
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
[
https://issues.apache.org/jira/browse/HBASE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730115#comment-16730115
]
xuqinya commented on HBASE-21656:
-
If this is a problem, I will add patch for it
> hbase-commom build failed on Windows7 using Cygwin
> ---
>
> Key: HBASE-21656
> URL: https://issues.apache.org/jira/browse/HBASE-21656
> Project: HBase
> Issue Type: Bug
>Reporter: xuqinya
>Priority: Major
>
> The command whoami and hostname returns the content followed by a
> character(\r),
> otherwise the generated file Version.java failed to compile.
> {code:java}
> @InterfaceAudience.Private
> public class Version {
> public static final String version = "3.0.0-SNAPSHOT";
> public static final String revision =
> "f32d2618430f70e1b0db92785294b2c7892cc02b";
> public static final String user = "xuqinya-PC\\xuqinya
> ";
> public static final String date = "Fri Dec 28 12:39:49 2018";
> public static final String url = "git://xuqinya-PC
> /cygdrive/e/gitwork/hbase";
> public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21656) hbase-commom build failed on Windows7 using Cygwin
xuqinya created HBASE-21656:
---
Summary: hbase-commom build failed on Windows7 using Cygwin
Key: HBASE-21656
URL: https://issues.apache.org/jira/browse/HBASE-21656
Project: HBase
Issue Type: Bug
Reporter: xuqinya
The command whoami and hostname returns the content followed by a character(\r),
otherwise the generated file Version.java failed to compile.
{code:java}
@InterfaceAudience.Private
public class Version {
public static final String version = "3.0.0-SNAPSHOT";
public static final String revision =
"f32d2618430f70e1b0db92785294b2c7892cc02b";
public static final String user = "xuqinya-PC\\xuqinya
";
public static final String date = "Fri Dec 28 12:39:49 2018";
public static final String url = "git://xuqinya-PC
/cygdrive/e/gitwork/hbase";
public static final String srcChecksum = "88f3dc17f75ffda6176faa649593b54e";
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
