[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-552968184 thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-552505302 what, you don't trust my code not to break your builds :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-552454018 @bgaborg : any final comments here? -this is needed to prevent thread leaks when S3A inits fail, and for other object stores. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-551790309 Rebased This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-550472609 just noticed there's a `TestFileSystemInitialization` suite. Moving everything there This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-547072586 Tests exist all good. tested S3 ireland This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-545585572 regression testing in progress. I haven't written any tests to explicitly explore the new codepaths. I will probably add a new common test which has a mock file system that fails during init and whose close() updates a static counter. Adding a test for S3A's explicit cleanup is a bit trickier. I could probably add an ITest which tried to create an FS with invalid settings -encryption ones get checked first- and then close() that after. This will at least show that the cleanup code is re-entrant. Trying to force a failure at a specific point after some closable items have been created (instrumentation, thread pools etc) and verifying that these are null afterwards is going to be very brittle. Avoid. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
