[
https://issues.apache.org/jira/browse/SQOOP-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Boglarka Egyed updated SQOOP-3383:
----------------------------------
Description:
FileSystem has a static cache meaning when the authentication happens in the
org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf method Sqoop
import will get the same FileSystem object from the cache thus its
authentication via the -D system properties is effectless. See
org.apache.hadoop.fs.FileSystem#get(java.net.URI,
org.apache.hadoop.conf.Configuration).
This static cache should be disabled (by settin {{fs.s3a.impl.disable.cache}}
to true) in the setup phase of the S3 tests to make sure Sqoop relies on the S3
credentials set via the -D system properties.
was:
In the current S3 test suite the setup phase includes a logic which sets the
AWS credentials in the Hadoop configuration in the
org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf method.
These credentials then are set via system properties calling the
org.apache.sqoop.testutil.S3TestUtils#getArgumentArrayBuilderForS3UnitTests
method too.
This is an unnecessary duplication that should be cleaned up to ease further
test implementation.
> Disable fs.s3a.impl.disable.cache in S3 tests
> ---------------------------------------------
>
> Key: SQOOP-3383
> URL: https://issues.apache.org/jira/browse/SQOOP-3383
> Project: Sqoop
> Issue Type: Sub-task
> Affects Versions: 1.4.7
> Reporter: Boglarka Egyed
> Assignee: Boglarka Egyed
> Priority: Minor
> Attachments: SQOOP-3383.patch
>
>
> FileSystem has a static cache meaning when the authentication happens in the
> org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf method
> Sqoop import will get the same FileSystem object from the cache thus its
> authentication via the -D system properties is effectless. See
> org.apache.hadoop.fs.FileSystem#get(java.net.URI,
> org.apache.hadoop.conf.Configuration).
> This static cache should be disabled (by settin {{fs.s3a.impl.disable.cache}}
> to true) in the setup phase of the S3 tests to make sure Sqoop relies on the
> S3 credentials set via the -D system properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
