[jira] [Updated] (HADOOP-16027) [DOC] Effective use of FS instances during S3A integration tests
[
https://issues.apache.org/jira/browse/HADOOP-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Mackrory updated HADOOP-16027:
---
Resolution: Fixed
Fix Version/s: 3.3.0
Status: Resolved (was: Patch Available)
> [DOC] Effective use of FS instances during S3A integration tests
>
>
> Key: HADOOP-16027
> URL: https://issues.apache.org/jira/browse/HADOOP-16027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-16027.001.patch, HADOOP-16027.002.patch
>
>
> While fixing HADOOP-15819 we found that a closed fs got into the static fs
> cache during testing, which caused other tests to fail when the tests were
> running sequentially.
> We should document some best practices in the testing section on the s3 docs
> with the following:
> {panel}
> Tests using FileSystems are fastest if they can recycle the existing FS
> instance from the same JVM. If you do that, you MUST NOT close or do unique
> configuration on them. If you want a guarantee of 100% isolation or an
> instance with unique config, create a new instance
> which you MUST close in the teardown to avoid leakage of resources.
> Do not add FileSystem instances (with e.g
> org.apache.hadoop.fs.FileSystem#addFileSystemForTesting) to the cache that
> will be modified or closed during the test runs. This can cause other tests
> to fail when using the same modified or closed FS instance. For more details
> see HADOOP-15819.
> {panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16027) [DOC] Effective use of FS instances during S3A integration tests
[
https://issues.apache.org/jira/browse/HADOOP-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-16027:
Attachment: HADOOP-16027.002.patch
> [DOC] Effective use of FS instances during S3A integration tests
>
>
> Key: HADOOP-16027
> URL: https://issues.apache.org/jira/browse/HADOOP-16027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-16027.001.patch, HADOOP-16027.002.patch
>
>
> While fixing HADOOP-15819 we found that a closed fs got into the static fs
> cache during testing, which caused other tests to fail when the tests were
> running sequentially.
> We should document some best practices in the testing section on the s3 docs
> with the following:
> {panel}
> Tests using FileSystems are fastest if they can recycle the existing FS
> instance from the same JVM. If you do that, you MUST NOT close or do unique
> configuration on them. If you want a guarantee of 100% isolation or an
> instance with unique config, create a new instance
> which you MUST close in the teardown to avoid leakage of resources.
> Do not add FileSystem instances (with e.g
> org.apache.hadoop.fs.FileSystem#addFileSystemForTesting) to the cache that
> will be modified or closed during the test runs. This can cause other tests
> to fail when using the same modified or closed FS instance. For more details
> see HADOOP-15819.
> {panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16027) [DOC] Effective use of FS instances during S3A integration tests
[
https://issues.apache.org/jira/browse/HADOOP-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-16027:
Attachment: HADOOP-16027.001.patch
> [DOC] Effective use of FS instances during S3A integration tests
>
>
> Key: HADOOP-16027
> URL: https://issues.apache.org/jira/browse/HADOOP-16027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-16027.001.patch
>
>
> While fixing HADOOP-15819 we found that a closed fs got into the static fs
> cache during testing, which caused other tests to fail when the tests were
> running sequentially.
> We should document some best practices in the testing section on the s3 docs
> with the following:
> {panel}
> Tests using FileSystems are fastest if they can recycle the existing FS
> instance from the same JVM. If you do that, you MUST NOT close or do unique
> configuration on them. If you want a guarantee of 100% isolation or an
> instance with unique config, create a new instance
> which you MUST close in the teardown to avoid leakage of resources.
> Do not add FileSystem instances (with e.g
> org.apache.hadoop.fs.FileSystem#addFileSystemForTesting) to the cache that
> will be modified or closed during the test runs. This can cause other tests
> to fail when using the same modified or closed FS instance. For more details
> see HADOOP-15819.
> {panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16027) [DOC] Effective use of FS instances during S3A integration tests
[
https://issues.apache.org/jira/browse/HADOOP-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-16027:
Status: Patch Available (was: In Progress)
Submitted patch v1. Just added a few lines to the doc, no need for an
additional test (justification).
> [DOC] Effective use of FS instances during S3A integration tests
>
>
> Key: HADOOP-16027
> URL: https://issues.apache.org/jira/browse/HADOOP-16027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-16027.001.patch
>
>
> While fixing HADOOP-15819 we found that a closed fs got into the static fs
> cache during testing, which caused other tests to fail when the tests were
> running sequentially.
> We should document some best practices in the testing section on the s3 docs
> with the following:
> {panel}
> Tests using FileSystems are fastest if they can recycle the existing FS
> instance from the same JVM. If you do that, you MUST NOT close or do unique
> configuration on them. If you want a guarantee of 100% isolation or an
> instance with unique config, create a new instance
> which you MUST close in the teardown to avoid leakage of resources.
> Do not add FileSystem instances (with e.g
> org.apache.hadoop.fs.FileSystem#addFileSystemForTesting) to the cache that
> will be modified or closed during the test runs. This can cause other tests
> to fail when using the same modified or closed FS instance. For more details
> see HADOOP-15819.
> {panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
