[
https://issues.apache.org/jira/browse/HADOOP-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570664#comment-16570664
]
Sean Mackrory commented on HADOOP-15546:
----------------------------------------
So with this small change I was able to get WASB and ABFS tests working. We
could leave the other properties as fs.azure.test because they should usually
be the same between WASB and ABFS. This is the only case I see where the values
MUST be different for the 2, and we don't have any way to specify 2 different
values at once unless we rename that property, so azure -> azure-bfs to resolve
the conflict. I don't think it's very clean, but the alternatives all seem
messier.
{code}
diff --git
a/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
b/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
index a2251ee..704b0b2 100644
---
a/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
+++
b/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
@@ -26,7 +26,7 @@
*/
@InterfaceStability.Evolving
public final class TestConfigurationKeys {
- public static final String FS_AZURE_TEST_ACCOUNT_NAME =
"fs.azure.test.account.name";
+ public static final String FS_AZURE_TEST_ACCOUNT_NAME =
"fs.azure-bfs.test.account.name";
public static final String FS_AZURE_TEST_ACCOUNT_KEY_PREFIX =
"fs.azure.test.account.key.";
public static final String FS_AZURE_TEST_HOST_NAME =
"fs.azure.test.host.name";
public static final String FS_AZURE_TEST_HOST_PORT =
"fs.azure.test.host.port";
{code}
This leaves me with a much cleaner list of test failures. Any thoughts on
including this in the current patch or resolving this conflict some other way?
* 3/5 tests fail in ITestWasbAbfsCompatibility and they look legit: 1 file
found when expecting 2, and 404's on some file paths.
* ITestAzureBlobFileSystemBackCompat also fails with a 404.
* ITestAzureBlobFileSystemRenameUnicode.testRenameFileUsingUnicode fails (gets
an empty listing)
* ITestAzureBlobFileSystemFileStatus.testEnsureStatusWorksForRoot and
ITestAzureBlobFileSystemListStatus.testListFiles both fail because of otherwise
identical instances of VersionedFileStatus and LocatedFileStatus not being
equal (only the type differs, and the version is null).
* 16/20 tests failed in ITestFileSystemOperationsWithThreads, with "No ...
operation found in logs"
I suspect the first 3 items are probably something server-side, so I'm going to
start looking into the bottom 2.
> ABFS: tune imports & javadocs; stabilise tests
> ----------------------------------------------
>
> Key: HADOOP-15546
> URL: https://issues.apache.org/jira/browse/HADOOP-15546
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: HADOOP-15407
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-15546-001.patch,
> HADOOP-15546-HADOOP-15407-001.patch, HADOOP-15546-HADOOP-15407-002.patch,
> HADOOP-15546-HADOOP-15407-003.patch, HADOOP-15546-HADOOP-15407-004.patch,
> HADOOP-15546-HADOOP-15407-005.patch, HADOOP-15546-HADOOP-15407-006.patch,
> HADOOP-15546-HADOOP-15407-006.patch, HADOOP-15546-HADOOP-15407-007.patch,
> HADOOP-15546-HADOOP-15407-008.patch, azure-auth-keys.xml
>
>
> Followup on HADOOP-15540 with some initial review tuning
> h2. Tuning
> * ordering of imports
> * rely on azure-auth-keys.xml to store credentials (change imports,
> docs,.gitignore)
> * log4j -> info
> * add a "." to the first sentence of all the javadocs I noticed.
> * remove @Public annotations except for some constants (which includes some
> commitment to maintain them).
> * move the AbstractFS declarations out of the src/test/resources XML file
> into core-default.xml for all to use
> * other IDE-suggested tweaks
> h2. Testing
> Review the tests, move to ContractTestUtil assertions, make more consistent
> to contract test setup, and general work to make the tests work well over
> slower links, document, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
