[
https://issues.apache.org/jira/browse/BEAM-12435?focusedWorklogId=620654&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-620654
]
ASF GitHub Bot logged work on BEAM-12435:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Jul/21 18:35
Start Date: 08/Jul/21 18:35
Worklog Time Spent: 10m
Work Description: mattrudary commented on a change in pull request #15036:
URL: https://github.com/apache/beam/pull/15036#discussion_r666431710
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/s3/S3FileSystemTest.java
##########
@@ -107,59 +106,64 @@ public static void afterClass() {
@Test
public void testGetScheme() {
- S3FileSystem s3FileSystem = new S3FileSystem(s3Options());
+ S3FileSystem s3FileSystem = new S3FileSystem(s3Config("s3"));
Review comment:
OK, I've done this as much as possible by adding alternate versions of
the tests called e.g. testGetSchemaWithS3Options. These follow the original
test code closely, although S3ResourceIds have to be constructed with an
explicit scheme of "s3" in these cases.
##########
File path:
sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/s3/S3FileSystemTest.java
##########
@@ -107,59 +106,64 @@ public static void afterClass() {
@Test
public void testGetScheme() {
- S3FileSystem s3FileSystem = new S3FileSystem(s3Options());
+ S3FileSystem s3FileSystem = new S3FileSystem(s3Config("s3"));
Review comment:
OK, I've done this as much as possible by adding alternate versions of
the tests called e.g. testGetSchemeWithS3Options. These follow the original
test code closely, although S3ResourceIds have to be constructed with an
explicit scheme of "s3" in these cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 620654)
Time Spent: 2h 10m (was: 2h)
> Generalize S3FileSystem
> -----------------------
>
> Key: BEAM-12435
> URL: https://issues.apache.org/jira/browse/BEAM-12435
> Project: Beam
> Issue Type: Improvement
> Components: io-java-aws
> Reporter: Matt Rudary
> Assignee: Matt Rudary
> Priority: P2
> Labels: aws, aws-s3
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> I'm working with multiple storage systems that speak the S3 api. I would like
> to support FileIO operations for these storage systems, but S3FileSystem
> hardcodes the s3 scheme (the various systems use different URI schemes) and
> it is in any case impossible to instantiate more than one in the current
> design.
> I'd like to refactor the code in org.apache.beam.sdk.io.aws.s3 (and maybe
> ...aws.options) somewhat to enable this use-case. I haven't worked out the
> details yet, but it will take some thought to make this work in a non-hacky
> way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
