[
https://issues.apache.org/jira/browse/PIG-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Slawski updated PIG-4950:
-------------------------------
Attachment: PIG-4950.1.patch
> Fix minor issues with running scripts in non-local FileSystems
> --------------------------------------------------------------
>
> Key: PIG-4950
> URL: https://issues.apache.org/jira/browse/PIG-4950
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.15.0, 0.16.0
> Reporter: Peter Slawski
> Priority: Minor
> Attachments: PIG-4950.1.patch
>
>
> There are two similar minor issues regarding running Pig scripts located in
> non-local FileSystems such as hdfs and s3.
>
> # The first occurs when the script path is passed using the ‘-f’ option. In
> this case, the script contents are not set in ScriptState. Instead a WARN
> message is logged due to an IOException being thrown. This is because the
> ‘remote’ path is treated as a local one. Instead, the path of the downloaded
> script should be passed over to ScriptState#setScript. As a result of this
> bug, an empty string is set for the “pig.script” property when the Pig job
> runs on a Hadoop cluster. Also, if Tez is being used, then the Dag info does
> not include the script contents as it normally does when a local script is
> passed.
> # The second issue is more minor, but #validateLogFile in the Main class is
> set to use the path given by the user rather than using the downloaded local
> file path. Again, #validateLogFile method treats the given path as a local
> one, but this would not be the case if the user specifies a remote path. i.e.
> one with a scheme such as hdfs or s3. This occurs in both cases: when the
> script is specified using the ‘-f’ option or when the script is passed as the
> last/remaining argument.
>
> Both fixes to these issues are to just pass in the local downloaded path
> instead. If the script path specified is a local one, then the local
> downloaded path would just be that path specified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
