[
https://issues.apache.org/jira/browse/PIG-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480944#comment-13480944
]
John Gordon commented on PIG-2796:
----------------------------------
After reviewing the patch, the overall codebase, and the abstraction in
question, I think we need another JIRA specifically to refactor temporary file
usage with abstract contexts.
This particular patch applies to a number of different temporary file
allocation patterns from pathname only to file+content. Additionally, there
are a number of temporary file allocations that work and aren't covered in this
JIRA. This JIRA gives us a better baseline to refactor and test on both
platforms, and I wouldn't want to do that refactor without a clean baseline to
pick up regressions.
I do agree that the abstraction should be more widely used for code health and
portability. I just don't think it should be blended with this patch, which
already touches a lot of files.
I created PIG-2995 to track this.
> Local temporary paths are not always valid HDFS path names.
> -----------------------------------------------------------
>
> Key: PIG-2796
> URL: https://issues.apache.org/jira/browse/PIG-2796
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Assignee: John Gordon
> Fix For: 0.11
>
> Attachments: 0006-Local-Remote-file-mapping-for-tests-with-temps.patch
>
>
> A number of pig scripts follow the pattern:
> File tempFile = File.createTempFile("this", ".txt");
> copyFromLocalToCluster (tempFile.to_string(), tempFile.to_string());
> tempFile.delete();
> The goal, here, seems to be to generate a temp filename to avoid issues on
> the next run if the file doesn't get cleaned up. The problem is that
> File.createTempFile on Windows creates files with names like
> C:\users\myuser\App data\local\temp\file.txt. The problem is that ":" is not
> a valid DFS character and so the put fails.
> The easy fix on this is to remove colons from the path before upload. Then
> we get something like C\users\myuser\App data\local\temp\file.txt which is a
> valid DFS pathname with minimal impact to the tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
