[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721174#comment-16721174
 ] 

Hudson commented on HBASE-20401:


Results for branch branch-1.3
[build #576 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576//General_Nightly_Build_Report/]


(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/576//JDK8_Nightly_Build_Report_(Hadoop2)/]




(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.3.3, 1.4.6, 2.2.0, 2.1.1, 2.0.2
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720784#comment-16720784
 ] 

Hudson commented on HBASE-20401:


SUCCESS: Integrated in Jenkins build HBase-1.3-IT #509 (See 
[https://builds.apache.org/job/HBase-1.3-IT/509/])
HBASE-20401 Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext 
(apurtell: rev 614b5f6e724db594b37d900d5b0fa4ada636eee5)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/LogCleaner.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileCleaner.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.3.3, 1.4.6, 2.2.0, 2.1.1, 2.0.2
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553796#comment-16553796
 ] 

Hudson commented on HBASE-20401:


Results for branch branch-1
[build #390 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/390/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/390//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/390//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/390//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553352#comment-16553352
 ] 

Hudson commented on HBASE-20401:


Results for branch branch-1.4
[build #394 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/394/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/394//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/394//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/394//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552877#comment-16552877
 ] 

Hudson commented on HBASE-20401:


Results for branch master
[build #405 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/405/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/405//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/405//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/405//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552676#comment-16552676
 ] 

Hudson commented on HBASE-20401:


Results for branch branch-2
[build #1015 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1015/]: 
(/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1015//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1015//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1015//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552628#comment-16552628
 ] 

Hudson commented on HBASE-20401:


Results for branch branch-2.0
[build #581 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/581/]: 
(/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/581//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/581//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/581//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552607#comment-16552607
 ] 

Hudson commented on HBASE-20401:


Results for branch branch-2.1
[build #93 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/93/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/93//console].




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/93//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/93//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552537#comment-16552537
 ] 

Reid Chan commented on HBASE-20401:
---

Unrelated failed UTs.
Pushed to branch-1 and branch-1.4.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552477#comment-16552477
 ] 

Hadoop QA commented on HBASE-20401:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m 
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
23s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
32s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
 9s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
46s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
35s{color} | {color:green} hbase-server: The patch generated 0 new + 14 
unchanged - 1 fixed = 14 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
13s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
2m  7s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}145m 52s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
22s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}190m  0s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestAssignmentManagerOnCluster |
|   | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
|   | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles |
|   | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-20401 |
| JIRA Patch URL | 

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552324#comment-16552324
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~reidchan] Thanks, it seems that I will have to submit a backport for this 
change in branch-1 although I attached as [^HBASE-20401.branch-1.002.patch] 
(this time I generated it with {{git format-patch}}), right?

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.master.001.patch, 
> HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, 
> HBASE-20401.master.004.patch, HBASE-20401.master.005.patch, 
> HBASE-20401.master.006.patch, HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552322#comment-16552322
 ] 

Reid Chan commented on HBASE-20401:
---

Let's wait for branch-1 QA results

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.master.001.patch, 
> HBASE-20401.master.002.patch, HBASE-20401.master.003.patch, 
> HBASE-20401.master.004.patch, HBASE-20401.master.005.patch, 
> HBASE-20401.master.006.patch, HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552226#comment-16552226
 ] 

Reid Chan commented on HBASE-20401:
---

Pushed to master, branch-2, branch-2.0, branch-2.1.
Thanks for you contributions. [~taklwu]
And hbase community recommends you to generate your patch using git-format so 
that your information can be included in commit message, please try it next 
time.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, 
> HBASE-20401.master.007.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551953#comment-16551953
 ] 

Hadoop QA commented on HBASE-20401:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  8m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue}  5m 
37s{color} | {color:blue} branch has no errors when building the reference 
guide. See footer for rendered docs, which you should manually inspect. {color} 
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
37s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m 
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  8m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  8m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue}  6m 
41s{color} | {color:blue} patch has no errors when building the reference 
guide. See footer for rendered docs, which you should manually inspect. {color} 
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
30s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
12m 54s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.7.4 or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  4m  
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}318m  6s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
 2s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}405m 59s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdminShell |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551785#comment-16551785
 ] 

Reid Chan commented on HBASE-20401:
---

nit:
{code}
 import java.util.concurrent.LinkedBlockingQueue;
 
+import java.util.concurrent.TimeUnit;
 import org.apache.hadoop.conf.Configuration;
{code}
should be
{code}
 import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.TimeUnit;

 import org.apache.hadoop.conf.Configuration;
{code}

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551433#comment-16551433
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~reidchan] for branch-1 patch, can we wait till HBASE-20559 get it first? 
because the dynamic configuration checking need to be added in this branch-1 
patch (then we don't need another JIRA, I think it could be done on Monday if 
[~zyork] reviews HBASE-20559 ;P ) 

meanwhile, for branch-2 and master, it's safe to commit.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550633#comment-16550633
 ] 

Reid Chan commented on HBASE-20401:
---

ping [~yuzhih...@gmail.com], [~mdrob], have more comments?

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550549#comment-16550549
 ] 

Hadoop QA commented on HBASE-20401:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  7m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
 1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue}  5m  
9s{color} | {color:blue} branch has no errors when building the reference 
guide. See footer for rendered docs, which you should manually inspect. {color} 
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
26s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m  
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  6m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue}  4m 
50s{color} | {color:blue} patch has no errors when building the reference 
guide. See footer for rendered docs, which you should manually inspect. {color} 
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
21s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
9m 43s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m  
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}245m  
5s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
 2s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}314m 26s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20401 |
| JIRA Patch URL | 

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550223#comment-16550223
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~reidchan] thanks for catching the {{Dynamic Configuration}} in the book 
chapter and it's been updated. RN has been updated based on your suggestion, 
and sorry that It's my first time writing the RN that may be too nerdy (will 
try to improve it over time :))

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch, HBASE-20401.master.006.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550155#comment-16550155
 ] 

Reid Chan commented on HBASE-20401:
---

Can the RN be more user friendly?
Those who doesn't read source code may feel hard to understand what those 
configurations are for and tune them for what..

Here is mine (as reference, it will be good if you have better),
{code}
When oldwals(hfile) cleaner cleans stale wals(hfiles), it will periodically 
check and wait  the clean results from filesystem, the total wait time will be 
no more than a max time.
The periodically wait and check configurations are ...
The max time configurations are ...
All support dynamic configuration.

...(Then here it is your tuning advice, please get rid of any code reference)...

{code}

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550126#comment-16550126
 ] 

Reid Chan commented on HBASE-20401:
---

Modified and new added configurations support {{Dynamic Configuration}, please 
update the hbase book as well.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550033#comment-16550033
 ] 

Hadoop QA commented on HBASE-20401:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  4m 
50s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
 1s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
43s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
13m 42s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}177m 
26s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}237m 27s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20401 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12932296/HBASE-20401.master.005.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux bbcfae429eae 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 
08:53:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / c92cda8420 |
| maven | version: Apache Maven 3.5.4 
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/13690/testReport/ |
| Max. process+thread count | 4779 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/13690/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Make `MAX_WAIT` and 

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550005#comment-16550005
 ] 

Hadoop QA commented on HBASE-20401:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
12s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
29s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
43s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
14s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
1m 55s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}135m 45s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
25s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 43s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles |
|   | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
|   | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-20401 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12932310/HBASE-20401.branch-1.001.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549840#comment-16549840
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~yuzhih...@gmail.com] I have attached the branch-1 patch, do I also need to 
attach branch-2 patch (it's a clean cherry-pick) ?

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549835#comment-16549835
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~mdrob] I modified the RN to include the use cases in LogCleaner.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549802#comment-16549802
 ] 

Mike Drob commented on HBASE-20401:
---

Sorry for the late review, can we add some docs for the when operators would 
want to tune these properties? Or expand on it in the RN.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, 
> HBASE-20401.master.005.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549140#comment-16549140
 ] 

Ted Yu commented on HBASE-20401:


[~taklwu]
Please fill out release note.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549136#comment-16549136
 ] 

Ted Yu commented on HBASE-20401:


lgtm
{code}
+if (cleanerThreadTimeoutMsec != this.cleanerThreadTimeoutMsec) {
+  this.cleanerThreadTimeoutMsec = cleanerThreadTimeoutMsec;
+}
{code}
Since there is no boolean flag to be set in this case, the if statement is not 
needed - just assign directly.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549126#comment-16549126
 ] 

Reid Chan commented on HBASE-20401:
---

Yes, i can, if you have no more comment and give +1. [~yuzhih...@gmail.com]

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549113#comment-16549113
 ] 

Ted Yu commented on HBASE-20401:


[~taklwu]:
Can you attach branch-1 patch ?

[~reidchan]:
Can you commit the patches once branch-1 patch passes QA bot ?

Thanks

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548757#comment-16548757
 ] 

Hadoop QA commented on HBASE-20401:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
37s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
42s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
11m 13s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}143m 
41s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}187m 45s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20401 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12932161/HBASE-20401.master.004.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 6b879fd5b882 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 8c85763327 |
| maven | version: Apache Maven 3.5.4 
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/13680/testReport/ |
| Max. process+thread count | 4074 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/13680/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Make `MAX_WAIT` and 

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548695#comment-16548695
 ] 

Reid Chan commented on HBASE-20401:
---

LGTM.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548681#comment-16548681
 ] 

Hadoop QA commented on HBASE-20401:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 5s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 7s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
9m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}195m 57s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}233m 51s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hbase.replication.TestSyncReplicationStandbyKillMaster |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20401 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12932154/HBASE-20401.master.003.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 89f5595b7a3f 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 
08:53:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 8c85763327 |
| maven | version: Apache Maven 3.5.4 
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/13677/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/13677/testReport/ |
| Max. process+thread count | 4827 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548509#comment-16548509
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~yuzhih...@gmail.com] I modified the elapsed duration calculation in both 
classes and updated, thanks.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch, HBASE-20401.master.004.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548428#comment-16548428
 ] 

Ted Yu commented on HBASE-20401:


{code}
  wait(waitIfNotFinished);
  waitTime += waitIfNotFinished;
{code}
The actual duration of wait may be different from waitIfNotFinished.
It would be better to compute the elapsed duration instead of using 
waitIfNotFinished directly in the last line above.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548429#comment-16548429
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~reidchan] I have done updated it with new property keys in {{{HFileCleaner}}} 
as well (also in the [PR#82|https://github.com/apache/hbase/pull/82]) and 
thanks for letting me know about only provide master branch for review, I'm 
wondered if {{HFileCleaner}} and {{LogCleaner }}can treat the same such that we 
have only introduce two property keys in {{CleanerChore}} only. let me know 
what you think.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, 
> HBASE-20401.master.003.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547349#comment-16547349
 ] 

Reid Chan commented on HBASE-20401:
---

BTW, provide patch for master branch first, and after +1, then provide patch 
for other branches.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547347#comment-16547347
 ] 

Reid Chan commented on HBASE-20401:
---

There's also a MAX_WAIT in HFileCleaner, can you work together?

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Tak Lon (Stephen) Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547239#comment-16547239
 ] 

Tak Lon (Stephen) Wu commented on HBASE-20401:
--

[~yuzhih...@gmail.com] thanks for reviewing this, I have attached master branch 
and other branches with adding OLD_WALS_** those constants

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.branch-1.002.patch, HBASE-20401.branch-2.001.patch, 
> HBASE-20401.master.001.patch, HBASE-20401.master.002.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547227#comment-16547227
 ] 

Ted Yu commented on HBASE-20401:


{code}
55static final long DEFAULT_CLEANER_THREAD_MAX_WAIT_MSEC = 60 * 1000;
{code}
Better add OLD_WALS_ to the identifier.
Same with the next DEFAULT constant.

> Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
> --
>
> Key: HBASE-20401
> URL: https://issues.apache.org/jira/browse/HBASE-20401
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
>  Labels: beginner
> Attachments: HBASE-20401.branch-1.001.patch, 
> HBASE-20401.master.001.patch
>
>
> When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls 
> CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for 
> notification (notify) from the fs.delete file thread. there might be two 
> situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished 
> when LogClearner call getResult.
>  # fs.delete never complete (strange but possible), then we need to wait for 
> a max of 60 seconds. here, 60 seconds might be too long
>  # getResult is waiting in the period of 500 milliseconds, but the fs.delete 
> has completed and setFromClear is set but yet notify(). one might want to 
> tune this 500 milliseconds to 200 or less .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)