[jira] [Updated] (HBASE-19633) Clean up the replication queues in the postPeerModification stage when removing a peer
[ https://issues.apache.org/jira/browse/HBASE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19633: -- Attachment: HBASE-19633-HBASE-19397-v2.patch Wrong patch. > Clean up the replication queues in the postPeerModification stage when > removing a peer > -- > > Key: HBASE-19633 > URL: https://issues.apache.org/jira/browse/HBASE-19633 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-19633-HBASE-19397-v1.patch, > HBASE-19633-HBASE-19397-v2.patch, HBASE-19633-HBASE-19397.patch > > > In the previous implementation, we can not always cleanly remove all the > replication queues when removing a peer since the removing work is done by RS > and if an RS is crashed then some queues may left there forever. That's why > we need to check if there are already some queues for a newly created peer > since we may reuse the peer id and causes problem. > With the new procedure based replication peer modification, I think we can do > it cleanly. After the RefreshPeerProcedures are done on all RSes, we can make > sure that no RS will create queue for this peer again, then we can iterate > over all the queues for all Rses and do another round of clean up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19633) Clean up the replication queues in the postPeerModification stage when removing a peer
[ https://issues.apache.org/jira/browse/HBASE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19633: -- Attachment: (was: HBASE-19633-HBASE-19397-v2.patch) > Clean up the replication queues in the postPeerModification stage when > removing a peer > -- > > Key: HBASE-19633 > URL: https://issues.apache.org/jira/browse/HBASE-19633 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-19633-HBASE-19397-v1.patch, > HBASE-19633-HBASE-19397.patch > > > In the previous implementation, we can not always cleanly remove all the > replication queues when removing a peer since the removing work is done by RS > and if an RS is crashed then some queues may left there forever. That's why > we need to check if there are already some queues for a newly created peer > since we may reuse the peer id and causes problem. > With the new procedure based replication peer modification, I think we can do > it cleanly. After the RefreshPeerProcedures are done on all RSes, we can make > sure that no RS will create queue for this peer again, then we can iterate > over all the queues for all Rses and do another round of clean up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19633) Clean up the replication queues in the postPeerModification stage when removing a peer
[ https://issues.apache.org/jira/browse/HBASE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19633: -- Attachment: HBASE-19633-HBASE-19397-v2.patch Moved the removeReplicatorIfQueueIsEmpty out. Scan two passes when removing all the queues for a peer, the reason is described in the comments. Also done some refactoring on the API of ReplicationPeers. > Clean up the replication queues in the postPeerModification stage when > removing a peer > -- > > Key: HBASE-19633 > URL: https://issues.apache.org/jira/browse/HBASE-19633 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-19633-HBASE-19397-v1.patch, > HBASE-19633-HBASE-19397-v2.patch, HBASE-19633-HBASE-19397.patch > > > In the previous implementation, we can not always cleanly remove all the > replication queues when removing a peer since the removing work is done by RS > and if an RS is crashed then some queues may left there forever. That's why > we need to check if there are already some queues for a newly created peer > since we may reuse the peer id and causes problem. > With the new procedure based replication peer modification, I think we can do > it cleanly. After the RefreshPeerProcedures are done on all RSes, we can make > sure that no RS will create queue for this peer again, then we can iterate > over all the queues for all Rses and do another round of clean up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307373#comment-16307373 ] Duo Zhang commented on HBASE-19369: --- I do not fully understand the problem here. Is the createNonRecursive method removed in hadoop3? > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch, HBASE-19369.v5.patch, HBASE-19369.v6.patch, > HBASE-19369.v7.patch, HBASE-19369.v8.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307360#comment-16307360
]
Hadoop QA commented on HBASE-19478:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
45s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19478 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904109/HBASE-19478.v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 412ae81c5141 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6c2aa4c9cc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10821/testReport/ |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10821/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL:
[jira] [Updated] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19478:
-
Attachment: HBASE-19478.v3.patch
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch, HBASE-19478.v2.patch,
> HBASE-19478.v3.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19680) BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations
Chia-Ping Tsai created HBASE-19680: -- Summary: BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations Key: HBASE-19680 URL: https://issues.apache.org/jira/browse/HBASE-19680 Project: HBase Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-2 Currently, BMI#mutate doesn't wait the result from AP so the errors are stored in AP. The only way which can return the errors to user is, calling the flush to catch the exception. That is non-intuitive. I feel BMI#mutate should wait the result. That is to say, user can parse the exception thrown by BM#mutate to get the failed mutations. Also, we can remove the global error from AP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19662) hbase-metrics-api fails checkstyle due to wrong import order
[
https://issues.apache.org/jira/browse/HBASE-19662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307317#comment-16307317
]
Chia-Ping Tsai commented on HBASE-19662:
Or use the full path instead.
{code}
${maven.checkstyle.version}
- org.apache.hbase
- hbase-checkstyle
- ${project.version}
-
-
com.puppycrawl.tools
checkstyle
${checkstyle.version}
-hbase/checkstyle.xml
-
hbase/checkstyle-suppressions.xml
+
hbase-checkstyle/src/main/resources/hbase/checkstyle.xml
+
hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
true
{code}
> hbase-metrics-api fails checkstyle due to wrong import order
>
>
> Key: HBASE-19662
> URL: https://issues.apache.org/jira/browse/HBASE-19662
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 19662.v1.txt
>
>
> In recent trunk builds, there were the following errors:
> {code}
> [ERROR]
> src/main/java/org/apache/hadoop/hbase/metrics/MetricRegistriesLoader.java:[31]
> (imports) ImportOrder: Wrong order for
> 'org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting'
> import.
> [ERROR]
> src/test/java/org/apache/hadoop/hbase/metrics/TestMetricRegistriesLoader.java:[28]
> (imports) ImportOrder: Wrong order for
> 'org.apache.hbase.thirdparty.com.google.common.collect.Lists' import.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19675) Miscellaneous HStore Class Improvements
[
https://issues.apache.org/jira/browse/HBASE-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307304#comment-16307304
]
Hadoop QA commented on HBASE-19675:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} hbase-server: The patch generated 0 new + 45
unchanged - 6 fixed = 45 total (was 51) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m
34s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}135m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19675 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904107/HBASE-19675.2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2f817ee97f08 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 360d465a4a |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10820/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10820/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Miscellaneous HStore Class
[jira] [Commented] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307303#comment-16307303 ] Chia-Ping Tsai commented on HBASE-19486: Could you make the fields be final? Otherwise LGTM > Periodically ensure records are not buffered too long by BufferedMutator > - > > Key: HBASE-19486 > URL: https://issues.apache.org/jira/browse/HBASE-19486 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19486-20171212-2117.patch, > HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, > HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, > HBASE-19486-20171219-1122-trigger-qa-run.patch, > HBASE-19486-20171220-1612-trigger-qa-run.patch, > HBASE-19486-20171220-2228-trigger-qa-run.patch, > HBASE-19486-20171223-1438-trigger-qa-run.patch, > HBASE-19486-20171223-1728-trigger-qa-run.patch, > HBASE-19486-20171223--trigger-qa-run.patch, > HBASE-19486-20171224-1101-trigger-qa-run.patch, > HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, > HBASE-19486-branch-1.v1.patch, HBASE-19486.20171231-105839-addendum.patch, > HBASE-19486.v0.patch > > > I'm working on several projects where we are doing stream / event type > processing instead of batch type processing. We mostly use Apache Flink and > Apache Beam for these projects. > When we ingest a continuous stream of events and feed that into HBase via a > BufferedMutator this all works fine. The buffer fills up at a predictable > rate and we can make sure it flushes several times per second into HBase by > tuning the buffer size. > We also have situations where the event rate is unpredictable. Some times > because the source is in reality a batch job that puts records into Kafka, > sometimes because it is the "predictable in production" application in our > testing environment (where only the dev triggers a handful of events). > For these kinds of use cases we need a way to 'force' the BufferedMutator to > automatically flush any records in the buffer even if the buffer is not full. > I'll put up a pull request with a proposed implementation for review against > the master (i.e. 3.0.0). > When approved I would like to backport this to the 1.x and 2.x versions of > the client in the same (as close as possible) way. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19677) Miscellaneous HFileCleaner Improvements
[
https://issues.apache.org/jira/browse/HBASE-19677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307300#comment-16307300
]
Hudson commented on HBASE-19677:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4320 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4320/])
HBASE-19677 Miscellaneous HFileCleaner Improvements (BELUGA BEHR) (tedyu: rev
360d465a4a22fa5457e475b7d62ba5fec5d0b27b)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
> Miscellaneous HFileCleaner Improvements
> ---
>
> Key: HBASE-19677
> URL: https://issues.apache.org/jira/browse/HBASE-19677
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19677.1.patch
>
>
> * Remove superfluous logging code guards
> * Introduce SLF4J parameter logging
> * Some simplifying:
> {code}
> -for (HFileDeleteTask task : largeFileQueue) {
> - leftOverTasks.add(task);
> -}
> -for (HFileDeleteTask task : smallFileQueue) {
> - leftOverTasks.add(task);
> -}
> +leftOverTasks.addAll(largeFileQueue);
> +leftOverTasks.addAll(smallFileQueue);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19678) HBase Admin security capabilities should be represented as a Set
[
https://issues.apache.org/jira/browse/HBASE-19678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307301#comment-16307301
]
Hudson commented on HBASE-19678:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4320 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4320/])
HBASE-19678 HBase Admin security capabilities should be represented as a
(tedyu: rev 6c2aa4c9ccea04ccf5c9c84de9677bd6232856e1)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
* (edit)
hbase-common/src/main/java/org/apache/hadoop/hbase/security/Superusers.java
> HBase Admin security capabilities should be represented as a Set
>
>
> Key: HBASE-19678
> URL: https://issues.apache.org/jira/browse/HBASE-19678
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19678.1.patch
>
>
> {code:title=org.apache.hadoop.hbase.client.Admin}
> /**
>* Return the set of supported security capabilities.
>* @throws IOException
>* @throws UnsupportedOperationException
>*/
> List getSecurityCapabilities() throws IOException;
> {code}
> The comment says a "set" but it returns a List. A Set would be the most
> appropriate data structure here, an immutable one perhaps, because the code
> that interacts with it looks up information using the _contains_ method which
> would be served well by a Set. Please change this interface to return a Set.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19678) HBase Admin security capabilities should be represented as a Set
[
https://issues.apache.org/jira/browse/HBASE-19678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19678:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-beta-2
Status: Resolved (was: Patch Available)
Thanks for the patch, BELUGA
> HBase Admin security capabilities should be represented as a Set
>
>
> Key: HBASE-19678
> URL: https://issues.apache.org/jira/browse/HBASE-19678
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19678.1.patch
>
>
> {code:title=org.apache.hadoop.hbase.client.Admin}
> /**
>* Return the set of supported security capabilities.
>* @throws IOException
>* @throws UnsupportedOperationException
>*/
> List getSecurityCapabilities() throws IOException;
> {code}
> The comment says a "set" but it returns a List. A Set would be the most
> appropriate data structure here, an immutable one perhaps, because the code
> that interacts with it looks up information using the _contains_ method which
> would be served well by a Set. Please change this interface to return a Set.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19678) HBase Admin security capabilities should be represented as a Set
[
https://issues.apache.org/jira/browse/HBASE-19678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19678:
---
Summary: HBase Admin security capabilities should be represented as a Set
(was: HBase Admin Security Capabilities Should Be Represented as a Set)
> HBase Admin security capabilities should be represented as a Set
>
>
> Key: HBASE-19678
> URL: https://issues.apache.org/jira/browse/HBASE-19678
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19678.1.patch
>
>
> {code:title=org.apache.hadoop.hbase.client.Admin}
> /**
>* Return the set of supported security capabilities.
>* @throws IOException
>* @throws UnsupportedOperationException
>*/
> List getSecurityCapabilities() throws IOException;
> {code}
> The comment says a "set" but it returns a List. A Set would be the most
> appropriate data structure here, an immutable one perhaps, because the code
> that interacts with it looks up information using the _contains_ method which
> would be served well by a Set. Please change this interface to return a Set.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19675) Miscellaneous HStore Class Improvements
[
https://issues.apache.org/jira/browse/HBASE-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-19675:
Status: Patch Available (was: Open)
> Miscellaneous HStore Class Improvements
> ---
>
> Key: HBASE-19675
> URL: https://issues.apache.org/jira/browse/HBASE-19675
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19675.1.patch, HBASE-19675.2.patch
>
>
> * Remove logging code guards in favor of slf4j parameters
> * Use {{CollectionsUtils.isEmpty()}} consistently
> * Small check-style fixes
> * Remove flow control logic from {{trace}} statement {code} if
> (LOG.isTraceEnabled()) {
> LOG.trace("No compacted files to archive");
> return;
> }{code}
> * Replace two calls to the same getter to ensure that the value doesn't
> change between calls {code} if (getCompactedFiles() != null) {
> for (HStoreFile file : getCompactedFiles()) {
> name2File.put(file.getFileInfo().getActiveFileName(), file);
> }
> }{code}
> * Make 'inputFiles' a Set for fast calls to {{contains}} method instead
> {code}//some of the input files might already be deleted
> List inputStoreFiles = new
> ArrayList<>(compactionInputs.size());
> for (HStoreFile sf : this.getStorefiles()) {
> if (inputFiles.contains(sf.getPath().getName())) {
> inputStoreFiles.add(sf);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19675) Miscellaneous HStore Class Improvements
[
https://issues.apache.org/jira/browse/HBASE-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-19675:
Attachment: HBASE-19675.2.patch
> Miscellaneous HStore Class Improvements
> ---
>
> Key: HBASE-19675
> URL: https://issues.apache.org/jira/browse/HBASE-19675
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19675.1.patch, HBASE-19675.2.patch
>
>
> * Remove logging code guards in favor of slf4j parameters
> * Use {{CollectionsUtils.isEmpty()}} consistently
> * Small check-style fixes
> * Remove flow control logic from {{trace}} statement {code} if
> (LOG.isTraceEnabled()) {
> LOG.trace("No compacted files to archive");
> return;
> }{code}
> * Replace two calls to the same getter to ensure that the value doesn't
> change between calls {code} if (getCompactedFiles() != null) {
> for (HStoreFile file : getCompactedFiles()) {
> name2File.put(file.getFileInfo().getActiveFileName(), file);
> }
> }{code}
> * Make 'inputFiles' a Set for fast calls to {{contains}} method instead
> {code}//some of the input files might already be deleted
> List inputStoreFiles = new
> ArrayList<>(compactionInputs.size());
> for (HStoreFile sf : this.getStorefiles()) {
> if (inputFiles.contains(sf.getPath().getName())) {
> inputStoreFiles.add(sf);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19675) Miscellaneous HStore Class Improvements
[
https://issues.apache.org/jira/browse/HBASE-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-19675:
Status: Open (was: Patch Available)
> Miscellaneous HStore Class Improvements
> ---
>
> Key: HBASE-19675
> URL: https://issues.apache.org/jira/browse/HBASE-19675
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19675.1.patch
>
>
> * Remove logging code guards in favor of slf4j parameters
> * Use {{CollectionsUtils.isEmpty()}} consistently
> * Small check-style fixes
> * Remove flow control logic from {{trace}} statement {code} if
> (LOG.isTraceEnabled()) {
> LOG.trace("No compacted files to archive");
> return;
> }{code}
> * Replace two calls to the same getter to ensure that the value doesn't
> change between calls {code} if (getCompactedFiles() != null) {
> for (HStoreFile file : getCompactedFiles()) {
> name2File.put(file.getFileInfo().getActiveFileName(), file);
> }
> }{code}
> * Make 'inputFiles' a Set for fast calls to {{contains}} method instead
> {code}//some of the input files might already be deleted
> List inputStoreFiles = new
> ArrayList<>(compactionInputs.size());
> for (HStoreFile sf : this.getStorefiles()) {
> if (inputFiles.contains(sf.getPath().getName())) {
> inputStoreFiles.add(sf);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19675) Miscellaneous HStore Class Improvements
[
https://issues.apache.org/jira/browse/HBASE-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-19675:
Summary: Miscellaneous HStore Class Improvements (was: Miscellaneous
HStore.java Improvements)
> Miscellaneous HStore Class Improvements
> ---
>
> Key: HBASE-19675
> URL: https://issues.apache.org/jira/browse/HBASE-19675
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19675.1.patch
>
>
> * Remove logging code guards in favor of slf4j parameters
> * Use {{CollectionsUtils.isEmpty()}} consistently
> * Small check-style fixes
> * Remove flow control logic from {{trace}} statement {code} if
> (LOG.isTraceEnabled()) {
> LOG.trace("No compacted files to archive");
> return;
> }{code}
> * Replace two calls to the same getter to ensure that the value doesn't
> change between calls {code} if (getCompactedFiles() != null) {
> for (HStoreFile file : getCompactedFiles()) {
> name2File.put(file.getFileInfo().getActiveFileName(), file);
> }
> }{code}
> * Make 'inputFiles' a Set for fast calls to {{contains}} method instead
> {code}//some of the input files might already be deleted
> List inputStoreFiles = new
> ArrayList<>(compactionInputs.size());
> for (HStoreFile sf : this.getStorefiles()) {
> if (inputFiles.contains(sf.getPath().getName())) {
> inputStoreFiles.add(sf);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19678) HBase Admin Security Capabilities Should Be Represented as a Set
[
https://issues.apache.org/jira/browse/HBASE-19678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307282#comment-16307282
]
BELUGA BEHR commented on HBASE-19678:
-
[~tedyu] It wasn't. It was just moved up a few lines instead of being buried
at the end of the method. Makes it more clear. I know I was surprised to find
it down there all alone.
> HBase Admin Security Capabilities Should Be Represented as a Set
>
>
> Key: HBASE-19678
> URL: https://issues.apache.org/jira/browse/HBASE-19678
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19678.1.patch
>
>
> {code:title=org.apache.hadoop.hbase.client.Admin}
> /**
>* Return the set of supported security capabilities.
>* @throws IOException
>* @throws UnsupportedOperationException
>*/
> List getSecurityCapabilities() throws IOException;
> {code}
> The comment says a "set" but it returns a List. A Set would be the most
> appropriate data structure here, an immutable one perhaps, because the code
> that interacts with it looks up information using the _contains_ method which
> would be served well by a Set. Please change this interface to return a Set.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19677) Miscellaneous HFileCleaner Improvements
[
https://issues.apache.org/jira/browse/HBASE-19677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19677:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-beta-2
Status: Resolved (was: Patch Available)
Thanks for the patch, BELUGA
> Miscellaneous HFileCleaner Improvements
> ---
>
> Key: HBASE-19677
> URL: https://issues.apache.org/jira/browse/HBASE-19677
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19677.1.patch
>
>
> * Remove superfluous logging code guards
> * Introduce SLF4J parameter logging
> * Some simplifying:
> {code}
> -for (HFileDeleteTask task : largeFileQueue) {
> - leftOverTasks.add(task);
> -}
> -for (HFileDeleteTask task : smallFileQueue) {
> - leftOverTasks.add(task);
> -}
> +leftOverTasks.addAll(largeFileQueue);
> +leftOverTasks.addAll(smallFileQueue);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307270#comment-16307270
]
Hadoop QA commented on HBASE-19478:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} hbase-backup: The patch generated 1 new + 24 unchanged
- 0 fixed = 25 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
44s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19478 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904105/HBASE-19478.v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1e6a9339be68 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd6050d09 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/10819/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10819/testReport/ |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10819/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Utilize multi-get to speed up WAL
[jira] [Updated] (HBASE-19677) Miscellaneous HFileCleaner Improvements
[
https://issues.apache.org/jira/browse/HBASE-19677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19677:
---
Summary: Miscellaneous HFileCleaner Improvements (was: Miscellaneous
HFileCleaner.java Improvements)
> Miscellaneous HFileCleaner Improvements
> ---
>
> Key: HBASE-19677
> URL: https://issues.apache.org/jira/browse/HBASE-19677
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-19677.1.patch
>
>
> * Remove superfluous logging code guards
> * Introduce SLF4J parameter logging
> * Some simplifying:
> {code}
> -for (HFileDeleteTask task : largeFileQueue) {
> - leftOverTasks.add(task);
> -}
> -for (HFileDeleteTask task : smallFileQueue) {
> - leftOverTasks.add(task);
> -}
> +leftOverTasks.addAll(largeFileQueue);
> +leftOverTasks.addAll(smallFileQueue);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19678) HBase Admin Security Capabilities Should Be Represented as a Set
[
https://issues.apache.org/jira/browse/HBASE-19678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307267#comment-16307267
]
Ted Yu commented on HBASE-19678:
Why was the following call removed ?
{code}
78 superUsers.add(currentUser);
{code}
> HBase Admin Security Capabilities Should Be Represented as a Set
>
>
> Key: HBASE-19678
> URL: https://issues.apache.org/jira/browse/HBASE-19678
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-19678.1.patch
>
>
> {code:title=org.apache.hadoop.hbase.client.Admin}
> /**
>* Return the set of supported security capabilities.
>* @throws IOException
>* @throws UnsupportedOperationException
>*/
> List getSecurityCapabilities() throws IOException;
> {code}
> The comment says a "set" but it returns a List. A Set would be the most
> appropriate data structure here, an immutable one perhaps, because the code
> that interacts with it looks up information using the _contains_ method which
> would be served well by a Set. Please change this interface to return a Set.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307265#comment-16307265
]
Hadoop QA commented on HBASE-19478:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} hbase-backup: The patch generated 1 new + 24 unchanged
- 0 fixed = 25 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
2s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19478 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904103/HBASE-19478.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux bd783d9e3cd0 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd6050d09 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/10818/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10818/testReport/ |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10818/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Utilize multi-get to speed up WAL file
[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307264#comment-16307264 ] stack commented on HBASE-19658: --- +1 please commit master and branch-2 [~anastas] > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307261#comment-16307261
]
Ted Yu commented on HBASE-19478:
lgtm
Pending QA result.
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch, HBASE-19478.v2.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307259#comment-16307259
]
Toshihiro Suzuki commented on HBASE-19478:
--
Thanks [~tedyu]. I just attached a new patch for the review.
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch, HBASE-19478.v2.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19478:
-
Attachment: HBASE-19478.v2.patch
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch, HBASE-19478.v2.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307253#comment-16307253
]
Ted Yu commented on HBASE-19478:
{code}
+ * Check if WAL file is eligible for deletion Future using multi-get
{code}
It seems 'Future' is not needed in above sentence.
{code}
+ public Map isWALFilesDeletable(Iterable
files) throws IOException {
{code}
For WALFiles, 'are' should be used - areWALFilesDeletable().
w.r.t. meaning of values in its returned Map, since you have this in the caller:
{code}
+boolean logInSystemTable = entry.getValue();
{code}
please modify the javadoc to match the actual meaning.
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17449) Add explicit document on different timeout settings
[ https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307250#comment-16307250 ] stack commented on HBASE-17449: --- Let's get this inwill be back > Add explicit document on different timeout settings > --- > > Key: HBASE-17449 > URL: https://issues.apache.org/jira/browse/HBASE-17449 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Yu Li >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0-beta-2 > > > Currently we have more than one timeout settings, mainly includes: > * hbase.rpc.timeout > * hbase.client.operation.timeout > * hbase.client.scanner.timeout.period > And in latest branch-1 or master branch code, we will have two other > properties: > * hbase.rpc.read.timeout > * hbase.rpc.write.timeout > However, in current refguid we don't have explicit instruction on the > difference of these timeout settings (there're explanations for each > property, but no instruction on when to use which) > In my understanding, for RPC layer timeout, or say each rpc call: > * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period > * Other operations: >1. For released versions: controlled by hbase.rpc.timeout >2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, > write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout > if the previous two are not set. > And hbase.client.operation.timeout is a higher-level control counting retry > in, or say the overall control for one user call. > After this JIRA, I hope when users ask questions like "What settings I should > use if I don't want to wait for more than 1 second for a single > put/get/scan.next call", we could give a neat answer. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17449) Add explicit document on different timeout settings
[ https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17449: -- Fix Version/s: (was: 2.0.0) 2.0.0-beta-2 > Add explicit document on different timeout settings > --- > > Key: HBASE-17449 > URL: https://issues.apache.org/jira/browse/HBASE-17449 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Yu Li >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0-beta-2 > > > Currently we have more than one timeout settings, mainly includes: > * hbase.rpc.timeout > * hbase.client.operation.timeout > * hbase.client.scanner.timeout.period > And in latest branch-1 or master branch code, we will have two other > properties: > * hbase.rpc.read.timeout > * hbase.rpc.write.timeout > However, in current refguid we don't have explicit instruction on the > difference of these timeout settings (there're explanations for each > property, but no instruction on when to use which) > In my understanding, for RPC layer timeout, or say each rpc call: > * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period > * Other operations: >1. For released versions: controlled by hbase.rpc.timeout >2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, > write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout > if the previous two are not set. > And hbase.client.operation.timeout is a higher-level control counting retry > in, or say the overall control for one user call. > After this JIRA, I hope when users ask questions like "What settings I should > use if I don't want to wait for more than 1 second for a single > put/get/scan.next call", we could give a neat answer. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17449) Add explicit document on different timeout settings
[ https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17449: -- Fix Version/s: 2.0.0 > Add explicit document on different timeout settings > --- > > Key: HBASE-17449 > URL: https://issues.apache.org/jira/browse/HBASE-17449 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Yu Li >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0-beta-2 > > > Currently we have more than one timeout settings, mainly includes: > * hbase.rpc.timeout > * hbase.client.operation.timeout > * hbase.client.scanner.timeout.period > And in latest branch-1 or master branch code, we will have two other > properties: > * hbase.rpc.read.timeout > * hbase.rpc.write.timeout > However, in current refguid we don't have explicit instruction on the > difference of these timeout settings (there're explanations for each > property, but no instruction on when to use which) > In my understanding, for RPC layer timeout, or say each rpc call: > * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period > * Other operations: >1. For released versions: controlled by hbase.rpc.timeout >2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, > write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout > if the previous two are not set. > And hbase.client.operation.timeout is a higher-level control counting retry > in, or say the overall control for one user call. > After this JIRA, I hope when users ask questions like "What settings I should > use if I don't want to wait for more than 1 second for a single > put/get/scan.next call", we could give a neat answer. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307248#comment-16307248
]
Toshihiro Suzuki commented on HBASE-19478:
--
I attached a patch that is added new method
BackupSystemTable#isWALFilesDeletable() using multi-get.
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19478:
-
Assignee: Toshihiro Suzuki
Status: Patch Available (was: Open)
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19478.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307244#comment-16307244 ] Ted Yu commented on HBASE-19658: +1 I ran TestCompactingToCellFlatMapMemStore with patch which passed. > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19359) Revisit the default config of hbase client retries number
[ https://issues.apache.org/jira/browse/HBASE-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307242#comment-16307242 ] stack edited comment on HBASE-19359 at 12/31/17 4:25 PM: - [~davelatham] thanks for chiming in. Poking around I think a bunch of the necessary work for an operation timeout is in place. Tests and doc is all that is missing. Let me work on finishing this up...hbase-17449 (a bunch of lads have done the work already) was (Author: stack): [~davelatham] thanks for chiming in. Poking around I think a bunch of the necessary work for an operation timeout is in place. Tests and doc is all that is missing. Let me work on finishing this up... > Revisit the default config of hbase client retries number > - > > Key: HBASE-19359 > URL: https://issues.apache.org/jira/browse/HBASE-19359 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19359.master.001.patch, > HBASE-19359.master.001.patch, HBASE-19359.master.001.patch > > > This should be sub-task of HBASE-19148. As the retries number effect too many > unit tests. So I open this issue to see the Hadoop QA result. > The default value of hbase.client.retries.number is 35. Plan to reduce this > to 10. > And for server side, the default hbase.client.serverside.retries.multiplier > is 10. So the server side retries number is 35 * 10 = 350. It is too big! > Plan to reduce hbase.client.serverside.retries.multiplier to 3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
[
https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19478:
-
Attachment: HBASE-19478.patch
> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> ---
>
> Key: HBASE-19478
> URL: https://issues.apache.org/jira/browse/HBASE-19478
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
> Attachments: HBASE-19478.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
> for (FileStatus file : files) {
> String wal = file.getPath().toString();
> boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production
> can get quite large.
> We should use multi-get to reduce the number of calls to backup table (which
> normally resides on another server).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19359) Revisit the default config of hbase client retries number
[ https://issues.apache.org/jira/browse/HBASE-19359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307242#comment-16307242 ] stack commented on HBASE-19359: --- [~davelatham] thanks for chiming in. Poking around I think a bunch of the necessary work for an operation timeout is in place. Tests and doc is all that is missing. Let me work on finishing this up... > Revisit the default config of hbase client retries number > - > > Key: HBASE-19359 > URL: https://issues.apache.org/jira/browse/HBASE-19359 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19359.master.001.patch, > HBASE-19359.master.001.patch, HBASE-19359.master.001.patch > > > This should be sub-task of HBASE-19148. As the retries number effect too many > unit tests. So I open this issue to see the Hadoop QA result. > The default value of hbase.client.retries.number is 35. Plan to reduce this > to 10. > And for server side, the default hbase.client.serverside.retries.multiplier > is 10. So the server side retries number is 35 * 10 = 350. It is too big! > Plan to reduce hbase.client.serverside.retries.multiplier to 3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19572) RegionMover should use the configured default port number and not the one from HConstants
[
https://issues.apache.org/jira/browse/HBASE-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307239#comment-16307239
]
Hadoop QA commented on HBASE-19572:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} hbase-server: The patch generated 0 new + 35
unchanged - 1 fixed = 35 total (was 36) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 1s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m
58s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}136m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19572 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904102/HBASE-19572.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8588b4daf475 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd6050d09 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10817/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10817/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> RegionMover should use the configured default port number and not the one
> from HConstants
>
[jira] [Updated] (HBASE-19572) RegionMover should use the configured default port number and not the one from HConstants
[
https://issues.apache.org/jira/browse/HBASE-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19572:
-
Status: Patch Available (was: Open)
> RegionMover should use the configured default port number and not the one
> from HConstants
> -
>
> Key: HBASE-19572
> URL: https://issues.apache.org/jira/browse/HBASE-19572
> Project: HBase
> Issue Type: Bug
>Reporter: Esteban Gutierrez
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19572.patch
>
>
> The issue I ran into HBASE-19499 was due RegionMover not using the port used
> by {{hbase-site.xml}}. The tool should use the value used in the
> configuration before falling back to the hardcoded value
> {{HConstants.DEFAULT_REGIONSERVER_PORT}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19572) RegionMover should use the configured default port number and not the one from HConstants
[
https://issues.apache.org/jira/browse/HBASE-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307216#comment-16307216
]
Toshihiro Suzuki commented on HBASE-19572:
--
I just attached a patch. [~esteban] if you have already started making a patch
for this Jira, please ignore the patch.
> RegionMover should use the configured default port number and not the one
> from HConstants
> -
>
> Key: HBASE-19572
> URL: https://issues.apache.org/jira/browse/HBASE-19572
> Project: HBase
> Issue Type: Bug
>Reporter: Esteban Gutierrez
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19572.patch
>
>
> The issue I ran into HBASE-19499 was due RegionMover not using the port used
> by {{hbase-site.xml}}. The tool should use the value used in the
> configuration before falling back to the hardcoded value
> {{HConstants.DEFAULT_REGIONSERVER_PORT}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-19572) RegionMover should use the configured default port number and not the one from HConstants
[
https://issues.apache.org/jira/browse/HBASE-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki reassigned HBASE-19572:
Assignee: Toshihiro Suzuki
> RegionMover should use the configured default port number and not the one
> from HConstants
> -
>
> Key: HBASE-19572
> URL: https://issues.apache.org/jira/browse/HBASE-19572
> Project: HBase
> Issue Type: Bug
>Reporter: Esteban Gutierrez
>Assignee: Toshihiro Suzuki
> Attachments: HBASE-19572.patch
>
>
> The issue I ran into HBASE-19499 was due RegionMover not using the port used
> by {{hbase-site.xml}}. The tool should use the value used in the
> configuration before falling back to the hardcoded value
> {{HConstants.DEFAULT_REGIONSERVER_PORT}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19572) RegionMover should use the configured default port number and not the one from HConstants
[
https://issues.apache.org/jira/browse/HBASE-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki updated HBASE-19572:
-
Attachment: HBASE-19572.patch
> RegionMover should use the configured default port number and not the one
> from HConstants
> -
>
> Key: HBASE-19572
> URL: https://issues.apache.org/jira/browse/HBASE-19572
> Project: HBase
> Issue Type: Bug
>Reporter: Esteban Gutierrez
> Attachments: HBASE-19572.patch
>
>
> The issue I ran into HBASE-19499 was due RegionMover not using the port used
> by {{hbase-site.xml}}. The tool should use the value used in the
> configuration before falling back to the hardcoded value
> {{HConstants.DEFAULT_REGIONSERVER_PORT}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[
https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307214#comment-16307214
]
Hadoop QA commented on HBASE-19658:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
5s{color} | {color:red} hbase-server: The patch generated 1 new + 18 unchanged
- 0 fixed = 19 total (was 18) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}128m 12s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}167m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRegionReplicaFailover |
| | hadoop.hbase.security.token.TestZKSecretWatcher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19658 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904099/HBASE-19658-V01.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3e688f392a72 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd6050d09 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/10815/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/10815/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10815/testReport/ |
|
[jira] [Commented] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator
[
https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307194#comment-16307194
]
Hadoop QA commented on HBASE-19486:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
16s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
49s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19486 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904100/HBASE-19486.20171231-105839-addendum.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0395b4d63311 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd6050d09 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10816/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/10816/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Periodically ensure records are not buffered too long by BufferedMutator
>
[jira] [Updated] (HBASE-19674) make_patch.sh version increment fails
[
https://issues.apache.org/jira/browse/HBASE-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niels Basjes updated HBASE-19674:
-
Description:
I have 5 things in the {{make_patch.sh}} script where I see room for
improvement:
1) BUG:
Assume my working branch is called {{HBASE-19673}}
Now if I run
{{dev-support/make_patch.sh -b origin/branch-1}}
a patch is created with the name
{{~/patches/HBASE-19673.v1.branch-1.patch}}
When I run the same command again the version is not incremented.
The reason is that the script checks for {{HBASE-19673.v1.patch}} which is
without the branch name.
2) Messy: The first patch created does NOT include the version tag at all.
3) Messy: The version starts with '1' so when we reach patch '10' they will be
ordered incorrectly in Jira (which is based on name)
4) New feature: I personally prefer using the timestamp as the 'version' of the
patch because these are much easier to order.
5) Messy: If you for example only have one file {{HBASE-19674.v05.patch}} then
the next file generated will be {{HBASE-19674.v01.patch}} instead of the
expected {{HBASE-19674.v06.patch}}
was:
I have 4 things in the {{make_patch.sh}} script where I see room for
improvement:
1) BUG:
My working branch is called {{HBASE-19673}}
Now if I run
{{dev-support/make_patch.sh -b origin/branch-1}}
a patch is created with the name
{{~/patches/HBASE-19673.v1.branch-1.patch}}
When I run the same command again the version is not incremented.
The reason is that the script checks for {{HBASE-19673.v1.patch}} which is
without the branch name.
2) Messy: The first patch created does NOT include the version tag at all.
3) Messy: The version starts with '1' so when we reach patch '10' they will be
ordered incorrectly in Jira (which is based on name)
4) Improve: I personally prefer using the timestamp as the 'version' of the
patch because these are much easier to order.
> make_patch.sh version increment fails
> -
>
> Key: HBASE-19674
> URL: https://issues.apache.org/jira/browse/HBASE-19674
> Project: HBase
> Issue Type: Improvement
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: HBASE-19674.20171230-131310.patch,
> HBASE-19674.20171230-152443.patch
>
>
> I have 5 things in the {{make_patch.sh}} script where I see room for
> improvement:
> 1) BUG:
> Assume my working branch is called {{HBASE-19673}}
> Now if I run
> {{dev-support/make_patch.sh -b origin/branch-1}}
> a patch is created with the name
> {{~/patches/HBASE-19673.v1.branch-1.patch}}
> When I run the same command again the version is not incremented.
> The reason is that the script checks for {{HBASE-19673.v1.patch}} which is
> without the branch name.
> 2) Messy: The first patch created does NOT include the version tag at all.
> 3) Messy: The version starts with '1' so when we reach patch '10' they will
> be ordered incorrectly in Jira (which is based on name)
> 4) New feature: I personally prefer using the timestamp as the 'version' of
> the patch because these are much easier to order.
> 5) Messy: If you for example only have one file {{HBASE-19674.v05.patch}}
> then the next file generated will be {{HBASE-19674.v01.patch}} instead of the
> expected {{HBASE-19674.v06.patch}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19673) Backport " Periodically ensure records are not buffered too long by BufferedMutator" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-19673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HBASE-19673: - Attachment: HBASE-19673.20171231-112539.branch-1.patch Backported the currently proposed thread safety changes. > Backport " Periodically ensure records are not buffered too long by > BufferedMutator" to branch-1 > > > Key: HBASE-19673 > URL: https://issues.apache.org/jira/browse/HBASE-19673 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HBASE-19673.20171230-130631.branch-1.patch, > HBASE-19673.20171230-131955.branch-1.patch, > HBASE-19673.20171231-112539.branch-1.patch > > > In HBASE-19486 the feature was built to periodically flush the > BufferedMutator. > Because backwards compatibility is important in the 1.x branch some > refactoring is needed to make this work. > As requested by [~chia7712] this separate issue was created -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HBASE-19486: - Status: Patch Available (was: Reopened) > Periodically ensure records are not buffered too long by BufferedMutator > - > > Key: HBASE-19486 > URL: https://issues.apache.org/jira/browse/HBASE-19486 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19486-20171212-2117.patch, > HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, > HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, > HBASE-19486-20171219-1122-trigger-qa-run.patch, > HBASE-19486-20171220-1612-trigger-qa-run.patch, > HBASE-19486-20171220-2228-trigger-qa-run.patch, > HBASE-19486-20171223-1438-trigger-qa-run.patch, > HBASE-19486-20171223-1728-trigger-qa-run.patch, > HBASE-19486-20171223--trigger-qa-run.patch, > HBASE-19486-20171224-1101-trigger-qa-run.patch, > HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, > HBASE-19486-branch-1.v1.patch, HBASE-19486.20171231-105839-addendum.patch, > HBASE-19486.v0.patch > > > I'm working on several projects where we are doing stream / event type > processing instead of batch type processing. We mostly use Apache Flink and > Apache Beam for these projects. > When we ingest a continuous stream of events and feed that into HBase via a > BufferedMutator this all works fine. The buffer fills up at a predictable > rate and we can make sure it flushes several times per second into HBase by > tuning the buffer size. > We also have situations where the event rate is unpredictable. Some times > because the source is in reality a batch job that puts records into Kafka, > sometimes because it is the "predictable in production" application in our > testing environment (where only the dev triggers a handful of events). > For these kinds of use cases we need a way to 'force' the BufferedMutator to > automatically flush any records in the buffer even if the buffer is not full. > I'll put up a pull request with a proposed implementation for review against > the master (i.e. 3.0.0). > When approved I would like to backport this to the 1.x and 2.x versions of > the client in the same (as close as possible) way. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HBASE-19486: - Attachment: HBASE-19486.20171231-105839-addendum.patch Made all WriteBufferPeriodicFlush related operations threadsafe by using AtomicLong instead of long and making the method that sets everything (setWriteBufferPeriodicFlush) to synchronized. [~chia7712] Please verify to check if I did it correctly/missed anything. > Periodically ensure records are not buffered too long by BufferedMutator > - > > Key: HBASE-19486 > URL: https://issues.apache.org/jira/browse/HBASE-19486 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19486-20171212-2117.patch, > HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, > HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, > HBASE-19486-20171219-1122-trigger-qa-run.patch, > HBASE-19486-20171220-1612-trigger-qa-run.patch, > HBASE-19486-20171220-2228-trigger-qa-run.patch, > HBASE-19486-20171223-1438-trigger-qa-run.patch, > HBASE-19486-20171223-1728-trigger-qa-run.patch, > HBASE-19486-20171223--trigger-qa-run.patch, > HBASE-19486-20171224-1101-trigger-qa-run.patch, > HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, > HBASE-19486-branch-1.v1.patch, HBASE-19486.20171231-105839-addendum.patch, > HBASE-19486.v0.patch > > > I'm working on several projects where we are doing stream / event type > processing instead of batch type processing. We mostly use Apache Flink and > Apache Beam for these projects. > When we ingest a continuous stream of events and feed that into HBase via a > BufferedMutator this all works fine. The buffer fills up at a predictable > rate and we can make sure it flushes several times per second into HBase by > tuning the buffer size. > We also have situations where the event rate is unpredictable. Some times > because the source is in reality a batch job that puts records into Kafka, > sometimes because it is the "predictable in production" application in our > testing environment (where only the dev triggers a handful of events). > For these kinds of use cases we need a way to 'force' the BufferedMutator to > automatically flush any records in the buffer even if the buffer is not full. > I'll put up a pull request with a proposed implementation for review against > the master (i.e. 3.0.0). > When approved I would like to backport this to the 1.x and 2.x versions of > the client in the same (as close as possible) way. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-19658: Status: Patch Available (was: Open) > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307185#comment-16307185 ] Anastasia Braginsky commented on HBASE-19658: - Thanks [~stack] for committing HBASE-19282 and assigning this one to me! Indeed a small miss of a line while merging with HBASE-19133. Fixed, testFlatteningToJumboCellChunkMap enabled, and a small patch attached here. If everything goes fine, should I commit it both in master and in branch-2? > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky updated HBASE-19658: Attachment: HBASE-19658-V01.patch > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19673) Backport " Periodically ensure records are not buffered too long by BufferedMutator" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-19673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HBASE-19673: - Status: Open (was: Patch Available) Needs the same threadsafe ensurances as in HBASE-19486 > Backport " Periodically ensure records are not buffered too long by > BufferedMutator" to branch-1 > > > Key: HBASE-19673 > URL: https://issues.apache.org/jira/browse/HBASE-19673 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HBASE-19673.20171230-130631.branch-1.patch, > HBASE-19673.20171230-131955.branch-1.patch > > > In HBASE-19486 the feature was built to periodically flush the > BufferedMutator. > Because backwards compatibility is important in the 1.x branch some > refactoring is needed to make this work. > As requested by [~chia7712] this separate issue was created -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307165#comment-16307165 ] Niels Basjes commented on HBASE-19486: -- Yes, will do. > Periodically ensure records are not buffered too long by BufferedMutator > - > > Key: HBASE-19486 > URL: https://issues.apache.org/jira/browse/HBASE-19486 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19486-20171212-2117.patch, > HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, > HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, > HBASE-19486-20171219-1122-trigger-qa-run.patch, > HBASE-19486-20171220-1612-trigger-qa-run.patch, > HBASE-19486-20171220-2228-trigger-qa-run.patch, > HBASE-19486-20171223-1438-trigger-qa-run.patch, > HBASE-19486-20171223-1728-trigger-qa-run.patch, > HBASE-19486-20171223--trigger-qa-run.patch, > HBASE-19486-20171224-1101-trigger-qa-run.patch, > HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, > HBASE-19486-branch-1.v1.patch, HBASE-19486.v0.patch > > > I'm working on several projects where we are doing stream / event type > processing instead of batch type processing. We mostly use Apache Flink and > Apache Beam for these projects. > When we ingest a continuous stream of events and feed that into HBase via a > BufferedMutator this all works fine. The buffer fills up at a predictable > rate and we can make sure it flushes several times per second into HBase by > tuning the buffer size. > We also have situations where the event rate is unpredictable. Some times > because the source is in reality a batch job that puts records into Kafka, > sometimes because it is the "predictable in production" application in our > testing environment (where only the dev triggers a handful of events). > For these kinds of use cases we need a way to 'force' the BufferedMutator to > automatically flush any records in the buffer even if the buffer is not full. > I'll put up a pull request with a proposed implementation for review against > the master (i.e. 3.0.0). > When approved I would like to backport this to the 1.x and 2.x versions of > the client in the same (as close as possible) way. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
