[
https://issues.apache.org/jira/browse/HADOOP-17784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17372617#comment-17372617
]
Steve Loughran commented on HADOOP-17784:
-----------------------------------------
Uh oh. This is our critical public dataset as it gives us CSV files with no
setup delays and AWS to pay the download charges.
We use it for testing
* random IO seek policy performance
* list performance with large directories
* S3 select.
This is going to be a disaster. We will have to see if we can find some other
public store with what we need (a large CSV file that's free to download)
There's always been options to override the path to the CSV file, which was
done to support private S3 store testing. We'll need to verify that switching
to this will allow old builds to keep testing happily. Some test suites do skip
if the target CSV file != that one in the constants (Region checking etc)
> hadoop-aws test bucket woul be deleted on Jul 1, 2021
> -----------------------------------------------------
>
> Key: HADOOP-17784
> URL: https://issues.apache.org/jira/browse/HADOOP-17784
> Project: Hadoop Common
> Issue Type: Test
> Components: fs/s3, test
> Reporter: Leona Yoda
> Priority: Major
>
> I found an anouncement that landsat-pds buket will be deleted on July 1, 2021
> (https://registry.opendata.aws/landsat-8/)
> and I think this bucket is used in th test of hadoop-aws module use
> [https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestConstants.java#L93]
>
> At this time I can access the bucket but we might have to change the test
> bucket in someday.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
