[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[
https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225674#comment-16225674
]
Chris Douglas commented on HDFS-9914:
-
Odd, {{TestWebHdfsTimeouts}} always fails for me. Though when I tried reverting
the patch, it still fails, so this isn't the cause.
> Fix configurable WebhDFS connect/read timeout
> -
>
> Key: HDFS-9914
> URL: https://issues.apache.org/jira/browse/HDFS-9914
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client, webhdfs
>Reporter: Xiaoyu Yao
>Assignee: Xiaoyu Yao
>Priority: Blocker
> Fix For: 2.9.0, 2.8.3, 3.0.0
>
> Attachments: HDFS-9914.001.patch, HDFS-9914.002.patch
>
>
> Webhdfs specific read/connect timeout as added HDFS-9887. This ticket is
> opened to fix the following issues in current implementation:
> 1. The webhdfs read/connect timeout should not affect connection for other
> callers of URLConnectionFactory.newSslConnConfigurator() such as
> QuorumJournalManager#QuorumJournalManger(), DFSck#DFSck() and
> TransferFsImage()
> 2. URLConnectionFactory#getSSLConnectionConfiguration() should honor webhdfs
> connect/read timeout even if any exception is thrown during customized SSL
> configuration.
>
> 3. OAuth2 webhdfs connection should honor the webhdfs connect/read timeout.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[ https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16224083#comment-16224083 ] Arpit Agarwal commented on HDFS-9914: - I ran ~20 iterations of this test in current trunk and it passed every time. Looks like TestWebHdfsTimeouts has been flaky for a while though (HDFS-11043). > Fix configurable WebhDFS connect/read timeout > - > > Key: HDFS-9914 > URL: https://issues.apache.org/jira/browse/HDFS-9914 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client, webhdfs >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Blocker > Fix For: 2.9.0, 2.8.3, 3.0.0 > > Attachments: HDFS-9914.001.patch, HDFS-9914.002.patch > > > Webhdfs specific read/connect timeout as added HDFS-9887. This ticket is > opened to fix the following issues in current implementation: > 1. The webhdfs read/connect timeout should not affect connection for other > callers of URLConnectionFactory.newSslConnConfigurator() such as > QuorumJournalManager#QuorumJournalManger(), DFSck#DFSck() and > TransferFsImage() > 2. URLConnectionFactory#getSSLConnectionConfiguration() should honor webhdfs > connect/read timeout even if any exception is thrown during customized SSL > configuration. > > 3. OAuth2 webhdfs connection should honor the webhdfs connect/read timeout. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[
https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16223272#comment-16223272
]
Chris Douglas commented on HDFS-9914:
-
Could this have caused failures in
{{org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts}}? The following error is
reproducible on trunk:
{noformat}
java.lang.AssertionError: Expected to find 'localhost:37905: connect timed out'
but got unexpected exception:java.net.SocketTimeoutException: localhost:37905:
Read timed out
{noformat}
> Fix configurable WebhDFS connect/read timeout
> -
>
> Key: HDFS-9914
> URL: https://issues.apache.org/jira/browse/HDFS-9914
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client, webhdfs
>Reporter: Xiaoyu Yao
>Assignee: Xiaoyu Yao
>Priority: Blocker
> Fix For: 2.9.0, 2.8.3, 3.0.0
>
> Attachments: HDFS-9914.001.patch, HDFS-9914.002.patch
>
>
> Webhdfs specific read/connect timeout as added HDFS-9887. This ticket is
> opened to fix the following issues in current implementation:
> 1. The webhdfs read/connect timeout should not affect connection for other
> callers of URLConnectionFactory.newSslConnConfigurator() such as
> QuorumJournalManager#QuorumJournalManger(), DFSck#DFSck() and
> TransferFsImage()
> 2. URLConnectionFactory#getSSLConnectionConfiguration() should honor webhdfs
> connect/read timeout even if any exception is thrown during customized SSL
> configuration.
>
> 3. OAuth2 webhdfs connection should honor the webhdfs connect/read timeout.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[ https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222511#comment-16222511 ] Hudson commented on HDFS-9914: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13147 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13147/]) HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by (arp: rev cd2a1dd0630e99e6696a9f0e8dc972604d38046a) * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java > Fix configurable WebhDFS connect/read timeout > - > > Key: HDFS-9914 > URL: https://issues.apache.org/jira/browse/HDFS-9914 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client, webhdfs >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Blocker > Fix For: 2.9.0, 2.8.3, 3.0.0 > > Attachments: HDFS-9914.001.patch, HDFS-9914.002.patch > > > Webhdfs specific read/connect timeout as added HDFS-9887. This ticket is > opened to fix the following issues in current implementation: > 1. The webhdfs read/connect timeout should not affect connection for other > callers of URLConnectionFactory.newSslConnConfigurator() such as > QuorumJournalManager#QuorumJournalManger(), DFSck#DFSck() and > TransferFsImage() > 2. URLConnectionFactory#getSSLConnectionConfiguration() should honor webhdfs > connect/read timeout even if any exception is thrown during customized SSL > configuration. > > 3. OAuth2 webhdfs connection should honor the webhdfs connect/read timeout. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[
https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221866#comment-16221866
]
Hadoop QA commented on HDFS-9914:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
58s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 48s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs-client: The patch
generated 0 new + 64 unchanged - 1 fixed = 64 total (was 65) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 48s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
50s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
31s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:5b98639 |
| JIRA Issue | HDFS-9914 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12894046/HDFS-9914.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux c335bbe129ef 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 36e158a |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/21832/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-HDFS-Build/21832/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client U:
hadoop-hdfs-project/hadoop-hdfs-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/21832/console |
| Powered by | Apac
[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[ https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220858#comment-16220858 ] Arpit Agarwal commented on HDFS-9914: - Thanks [~xyao]. +1 for the v2 patch, pending Jenkins. I've kicked off a new pre-commit run for this issue. > Fix configurable WebhDFS connect/read timeout > - > > Key: HDFS-9914 > URL: https://issues.apache.org/jira/browse/HDFS-9914 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HDFS-9914.001.patch, HDFS-9914.002.patch > > > Webhdfs specific read/connect timeout as added HDFS-9887. This ticket is > opened to fix the following issues in current implementation: > 1. The webhdfs read/connect timeout should not affect connection for other > callers of URLConnectionFactory.newSslConnConfigurator() such as > QuorumJournalManager#QuorumJournalManger(), DFSck#DFSck() and > TransferFsImage() > 2. URLConnectionFactory#getSSLConnectionConfiguration() should honor webhdfs > connect/read timeout even if any exception is thrown during customized SSL > configuration. > > 3. OAuth2 webhdfs connection should honor the webhdfs connect/read timeout. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout
[
https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184000#comment-15184000
]
Hadoop QA commented on HDFS-9914:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 14s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client: patch generated 1
new + 66 unchanged - 0 fixed = 67 total (was 66) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 56s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client generated 1 new +
0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hadoop-hdfs-client in the patch passed with JDK
v1.8.0_74. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hadoop-hdfs-client in the patch passed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 36s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-project/hadoop-hdfs-client |
| | Private method
org.apache.hadoop.hdfs.web.URLConnectionFactory.newSslConnConfigurator(int,
Configuration) is never called At URLConnectionFactory.java:never called At
URLConnectionFactory.java:[line 161] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca
