[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655067#comment-16655067 ] Adam Antal commented on HADOOP-11100: - Thanks for the help and review [~knanasi] and [~xiaochen]! > Support to configure ftpClient.setControlKeepAliveTimeout > -- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Fix For: 3.3.0 > > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HADOOP-11100.004.patch, HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654054#comment-16654054 ] Hudson commented on HADOOP-11100: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15244 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15244/]) HADOOP-11100. Support to configure ftpClient.setControlKeepAliveTimeout. (xiao: rev 24dc068a361648b4e59e1807b07ff2239f41c740) * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/ftp/TestFTPFileSystem.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java * (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml > Support to configure ftpClient.setControlKeepAliveTimeout > -- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Fix For: 3.3.0 > > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HADOOP-11100.004.patch, HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651465#comment-16651465 ] Hadoop QA commented on HADOOP-11100: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 2s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 26s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 18s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 46s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}100m 9s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 | | JIRA Issue | HADOOP-11100 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12944073/HADOOP-11100.004.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 7bf999e45bce 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 0bf8a11 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_181 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/15373/testReport/ | | Max. process+thread count | 1385 (vs. ulimit of 1) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/15373/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automaticall
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651277#comment-16651277 ] Adam Antal commented on HADOOP-11100: - Thanks [~xiaochen] for the valuable insight. I added only the fs.ftp.timeout config (defaulting to zero), as it is the current behaviour. (note that I removed fs.ftp.timeout.enabled since we don't need it). Also modified the core-default.xml and {{TestCommonConfigurationFields}} as you suggested. Pending jenkins for v4. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HADOOP-11100.004.patch, HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648244#comment-16648244 ] Xiao Chen commented on HADOOP-11100: Thanks [~adam.antal] for the good work here, and [~knanasi] for the review. Overall LGTM. 2 additional comments: * We should add a core-default.xml entry for the new config. \{{TestCommonConfigurationFields}} usually catches this, but ftp is kind of a special case so missed here. Once the config is added to core-default.xml, you'd need to update \{{TestCommonConfigurationFields}} to make the test pass. * Adding a timeout is a nice improvement. Strictly speaking, setting the default changes the current behavior. In general we try not to change the default behavior, for compatibility reasons. Only when something is obviously wrong, to the point which we consider it a bug, would we change the default behavior. I'd suggest we set the default to 0 here, to retain existing behavior (since FTPFileSystem has been there for a long time and is marked as Public Stable). > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647794#comment-16647794 ] Kitti Nanasi commented on HADOOP-11100: --- Thanks for the answer [~adam.antal]! You are right, I double checked the javadoc and the parameter is in seconds. I don't have any reason against setting this timeout to 5 minutes if it works fine with other FTP configs. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647666#comment-16647666 ] Adam Antal commented on HADOOP-11100: - Thanks for the reviewing, [~knanasi]. I just forget to answer one of your question. Although I did not find it surely, I believe the FTPClient:setControlKeepAliveTimeout() function requires input as second and not milliseconds. (In that function the input got multiplied by 1000, so in the inside structure stores the value as milliseconds, but the input is expected to in seconds.) It seems 300 seconds (5 min) commonly used as default timeout value across various FTP configs, but this is just an arbitrary value, and can be modified. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647588#comment-16647588 ] Kitti Nanasi commented on HADOOP-11100: --- Thanks for the new patch [~adam.antal]! It looks good to me. +1 > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646517#comment-16646517 ] Hadoop QA commented on HADOOP-11100: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 45s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 30s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 10s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 43s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 41s{color} | {color:red} The patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 99m 22s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 | | JIRA Issue | HADOOP-11100 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12943424/HADOOP-11100.003.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux d004c3fdb0dd 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug 17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8a37983 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_181 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/15350/testReport/ | | asflicense | https://builds.apache.org/job/PreCommit-HADOOP-Build/15350/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 1467 (vs. ulimit of 1) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/15350/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Support to configure ftp
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646384#comment-16646384 ] Adam Antal commented on HADOOP-11100: - Thanks [~knanasi] for looking into the patch. Also thanks for pointing me to the right direction about the testing, it looks straightforward now. To be able to be tested, I moved the functionality into a separate function, and I added the tests into patch v3. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HADOOP-11100.003.patch, > HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643708#comment-16643708 ] Kitti Nanasi commented on HADOOP-11100: --- Thanks for working on this [~adam.antal]! You can test the added configuration in TestFTPFileSystem, like in the test TestFTPFileSystem#testFTPTransferMode. Do I understand correctly that the default timeout is 300 milliseconds? > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643441#comment-16643441 ] Hadoop QA commented on HADOOP-11100: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 15s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 22m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 12s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 28s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 55s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}109m 31s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 | | JIRA Issue | HADOOP-11100 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12942999/HADOOP-11100.002.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b2329bb75cc8 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 7ba1cfd | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_181 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/15321/testReport/ | | Max. process+thread count | 1347 (vs. ulimit of 1) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/15321/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Support to
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643100#comment-16643100 ] Adam Antal commented on HADOOP-11100: - Uploaded patch v2. Added "fs.ftp.timeout.enabled" to config whether the timeout should be set. The change only affected {code:java} private FTPClient connect() throws IOException {code} method, which is private, so it's not intended to be tested directly. Other way I can think the timeout could be tested is some mocking approach, but it needs refactoring, and we did not test that function before either. In short, IMO testing if not applicable. As a first guess 5 min seems ok for default value - can be modified later. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643087#comment-16643087 ] Adam Antal commented on HADOOP-11100: - Misnamed previous patch. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HADOOP-11100.002.patch, HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638646#comment-16638646 ] Hadoop QA commented on HADOOP-11100: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 26s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 6s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 24s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 40s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 94m 23s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 | | JIRA Issue | HADOOP-11100 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12942412/HDFS-11000.001.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux d5152de80d00 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15 17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 81f635f | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_181 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/15290/testReport/ | | Max. process+thread count | 1359 (vs. ulimit of 1) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/15290/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Support to co
[jira] [Commented] (HADOOP-11100) Support to configure ftpClient.setControlKeepAliveTimeout
[ https://issues.apache.org/jira/browse/HADOOP-11100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638480#comment-16638480 ] Adam Antal commented on HADOOP-11100: - Uploaded patch v1, incorporating the changes you suggested in the first place. Following issues: * I don't see how could this function be tested. * Default has been set to 5 min. Is this reasonable? * In my opinion in order to be compatible with the current behaviour this feature needs to be turned off by default. > Support to configure ftpClient.setControlKeepAliveTimeout > --- > > Key: HADOOP-11100 > URL: https://issues.apache.org/jira/browse/HADOOP-11100 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.3.0 >Reporter: Krishnamoorthy Dharmalingam >Assignee: Adam Antal >Priority: Minor > Attachments: HDFS-11000.001.patch > > > In FTPFilesystem or Configuration, timeout is not possible to configure. > It is very straight forward to configure, in FTPFilesystem.connect() method. > ftpClient.setControlKeepAliveTimeout > Like > private FTPClient connect() throws IOException { > ... > String timeout = conf.get("fs.ftp.timeout." + host); > ... > ftpClient.setControlKeepAliveTimeout(new Integer(300)); > > } -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org