[
https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975281#comment-13975281
]
Steve Loughran commented on HADOOP-9361:
----------------------------------------
linking as a dependent on HDFS-4258 -handling of rename during operations on
open files -including append-
Without this, a test of HDFS append+ rename fails:
{code}
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSAppendContract
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.481 sec <<<
FAILURE! - in org.apache.hadoop.fs.contract.hdfs.TestHDFSAppendContract
testRenameFileBeingAppended(org.apache.hadoop.fs.contract.hdfs.TestHDFSAppendContract)
Time elapsed: 0.044 sec <<< FAILURE!
java.lang.AssertionError: renamed destination file does not exist: not found
hdfs://localhost:54005/test/test/renamed in hdfs://localhost:54005/test/test
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertPathExists(ContractTestUtils.java:587)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.assertPathExists(AbstractFSContractTestBase.java:254)
at
org.apache.hadoop.fs.contract.AbstractAppendContractTest.testRenameFileBeingAppended(AbstractAppendContractTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
{code}
> Strictly define the expected behavior of filesystem APIs and write tests to
> verify compliance
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-9361
> URL: https://issues.apache.org/jira/browse/HADOOP-9361
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
> Affects Versions: 3.0.0, 2.2.0, 2.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-9361-001.patch, HADOOP-9361-002.patch,
> HADOOP-9361-003.patch, HADOOP-9361-004.patch, HADOOP-9361-005.patch,
> HADOOP-9361-006.patch, HADOOP-9361-007.patch, HADOOP-9361-008.patch,
> HADOOP-9361-009.patch, HADOOP-9361-011.patch
>
>
> {{FileSystem}} and {{FileContract}} aren't tested rigorously enough -while
> HDFS gets tested downstream, other filesystems, such as blobstore bindings,
> don't.
> The only tests that are common are those of {{FileSystemContractTestBase}},
> which HADOOP-9258 shows is incomplete.
> I propose
> # writing more tests which clarify expected behavior
> # testing operations in the interface being in their own JUnit4 test classes,
> instead of one big test suite.
> # Having each FS declare via a properties file what behaviors they offer,
> such as atomic-rename, atomic-delete, umask, immediate-consistency -test
> methods can downgrade to skipped test cases if a feature is missing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
