[
https://issues.apache.org/jira/browse/HDFS-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14968785#comment-14968785
]
Steve Loughran commented on HDFS-9263:
--------------------------------------
Chris, the test/build/data path appeared throughout the code as the default
value if the sysprop wasn't set, it also appeared elsewhere. Consolidating it
means that there is one place for the default. As you note, this shouldn't be
the cause of the leftovers. Doing a bulk search for the path means that we are
guaranteed to have found the uses; consolidation means it won't ever arise
again.
Going through that code shows that a fair few of the tests have hard-coded
expectations of the path set up by minidfs cluster, specifically
${test.build.data}/dfs. That's going to be an intermittent breaker of parallel
test runs, as minidfs clusters may come up on separate ports, but they could be
stamping on each other's datasets. We hadn't really spotted that to date (and
the audits wont show it), but it's there.
If we not only consolidate test dir setup, but do it in a way that isolates it
for each test suite, we get that isolation. Trouble is, the -001 patch does
give isolation for every MiniDFS cluster, but breaks those tests which expect
restarted DFS clusters to have the same file path. We're going to need to come
up with the unique path for each test suite or method, not per cluster instance
—or at least add an option for that.
> tests are using /test/build/data; breaking Jenkins
> --------------------------------------------------
>
> Key: HDFS-9263
> URL: https://issues.apache.org/jira/browse/HDFS-9263
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0
> Environment: Jenkins
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Attachments: HDFS-9263-001.patch
>
>
> Some of the HDFS tests are using the path {{test/build/data}} to store files,
> so leaking files which fail the new post-build RAT test checks on Jenkins
> (and dirtying all development systems with paths which {{mvn clean}} will
> miss.
> fix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
