[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394873#comment-14394873
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2102 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2102/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394820#comment-14394820
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #153 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/153/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394520#comment-14394520
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-Hdfs-trunk #2084 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2084/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394505#comment-14394505
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #143 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/143/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394353#comment-14394353
]
Jean-Pierre Matsumoto commented on HADOOP-9805:
---
You're welcome. I'm glad to contribute.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394333#comment-14394333
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-Yarn-trunk #886 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/886/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394311#comment-14394311
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #152 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/152/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
* hadoop-common-project/hadoop-common/CHANGES.txt
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393732#comment-14393732
]
Hudson commented on HADOOP-9805:
FAILURE: Integrated in Hadoop-trunk-Commit #7499 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7499/])
HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
Contributed by Jean-Pierre Matsumoto. (cnauroth: rev
5763b173d34dcf7372520076f00b576f493662cd)
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/rawlocal/TestRawlocalContractRename.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
* hadoop-common-project/hadoop-common/CHANGES.txt
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391882#comment-14391882
]
Hadoop QA commented on HADOOP-9805:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708851/HADOOP-9805.003.patch
against trunk revision 75cb1d4.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6046//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6046//console
This message is automatically generated.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391823#comment-14391823
]
Colin Patrick McCabe commented on HADOOP-9805:
--
+1 from me. Will hold off a bit in case others have comments.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch,
> HADOOP-9805.003.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391612#comment-14391612
]
Chris Nauroth commented on HADOOP-9805:
---
It's unfortunate that the new method has to run {{pathToFile}} on the source
and destination again. There is quite a bit of work behind that method call,
and the result is already available in the scope of {{rename}}.
I suggest that the new method accept the arguments as both {{Path}} and
{{File}} to avoid the redundant call. Combining that with Colin's rename
suggestion, the signature would be:
{code}
public final boolean handleEmptyDstDirectoryOnWindows(Path src, File srcFile,
Path dst, File dstFile) throws IOException
{code}
Admittedly, that's a strange method signature, but this is only meant to be
visible for testing, not as a public API. To that end, let's also mark the
method {{final}}.
bq. Of course this bug is Windows-specific and we could easily fix it in a
follow-on JIRA (probably via JNI code).
I'm not sure this is true. Last time I checked, it looked like the Windows API
did not offer any way to match the POSIX semantics exactly. I'd be happy to be
proven wrong though, because the current approach is admittedly not atomic. :-)
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391495#comment-14391495
]
Colin Patrick McCabe commented on HADOOP-9805:
--
Looks good.
How about changing "renamePOSIX" to be named "handleEmptyDstDirectoryOnWindows"?
"renamePOSIX" sounds like something that should be called on POSIX platforms,
not on Windows. Also, it doesn't suggest that its use case is handling empty
destination directories.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387570#comment-14387570
]
Hadoop QA commented on HADOOP-9805:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708249/HADOOP-9805.002.patch
against trunk revision cc0a01c.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6027//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6027//console
This message is automatically generated.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch, HADOOP-9805.002.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387356#comment-14387356
]
Colin Patrick McCabe commented on HADOOP-9805:
--
{code}
366 // Enforce POSIX rename behavior that a source directory replaces an
367 // existing destination if the destination is an empty directory.
On most
368 // platforms, this is already handled by the Java API call above.
Some
369 // platforms (notably Windows) do not provide this behavior, so the
Java API
370 // call renameTo(dstFile) fails. Delete destination and attempt
rename
371 // again.
{code}
This code should not be needed on UNIX, right? This should be inside an {{if
(Shell.WINDOWS)}} block.
This also contains a small race condition where if the rename fails due to an
existing file, but that file is deleted by another thread in between the rename
and the call to {{File#exists}}, we will erroneously return {{false}}. Of
course this bug is Windows-specific and we could easily fix it in a follow-on
JIRA (probably via JNI code).
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385736#comment-14385736
]
Steve Loughran commented on HADOOP-9805:
LGTM. Chris: what do you think?
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385596#comment-14385596
]
Hadoop QA commented on HADOOP-9805:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708016/HADOOP-9805.001.patch
against trunk revision 3d9132d.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6021//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6021//console
This message is automatically generated.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-9805.001.patch
>
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14369672#comment-14369672
]
Jean-Pierre Matsumoto commented on HADOOP-9805:
---
Thanks for the input. But I found more calls to
{{RawLocalFileSystem.rename(Path,Path)}} than
{{TestNativeIO.renameTo(File,File)}}. I hope my patch will be a bit useful.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368833#comment-14368833
]
Steve Loughran commented on HADOOP-9805:
Now that Hadoop is 2.7+ only the new nativeio operations can be used for rename
& other things, which fail better
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Assignee: Jean-Pierre Matsumoto
>Priority: Minor
> Labels: newbie
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9805) Refactor RawLocalFileSystem#rename for improved testability.
[
https://issues.apache.org/jira/browse/HADOOP-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725457#comment-13725457
]
Chris Nauroth commented on HADOOP-9805:
---
If the fallback logic was refactored into a static {{VisibleForTesting}}
method, then we could write tests to cover that code on any platform. Thanks
to [~ste...@apache.org] for the idea.
> Refactor RawLocalFileSystem#rename for improved testability.
>
>
> Key: HADOOP-9805
> URL: https://issues.apache.org/jira/browse/HADOOP-9805
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0, 1-win, 1.3.0, 2.1.1-beta
>Reporter: Chris Nauroth
>Priority: Minor
>
> {{RawLocalFileSystem#rename}} contains fallback logic to provide POSIX rename
> behavior on platforms where {{java.io.File#renameTo}} fails. The method
> returns early if {{java.io.File#renameTo}} succeeds, so test runs may not
> cover the fallback logic depending on the platform.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
