[
https://issues.apache.org/jira/browse/PIG-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073649#comment-13073649
]
Thejas M Nair commented on PIG-2198:
------------------------------------
':' is what is used for separating path names in linux (including java class
path), so that is what is intuitive to users on linux. Also, ';' is
interpreted by (bash) shell as end of command, users will have to use quotes to
avoid that interpretation.
I think it is better to make the delimiter configurable. Another option is to
not use ':' within quoted strings as delimiters, but getting those quotes
through the shell to pig would require escaping the quotes, and that would be
ugly.
> pig.additional.jars does not support s3 paths
> ---------------------------------------------
>
> Key: PIG-2198
> URL: https://issues.apache.org/jira/browse/PIG-2198
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Reporter: Jeremy Custenborder
> Priority: Minor
> Attachments: PIG-2198.patch
>
>
> The delimiter for additional jars is ":" which is part of the uri for a S3
> path.
> *Repro*
> Add the following to the command line.
> {quote}
> -Dpig.additional.jars=s3://bucket/udf.jar
> {quote}
> {quote}
> Error before Pig is launched
> ----------------------------
> ERROR 101: Could not copy s3 to local destination
> /mnt/var/lib/hadoop/steps/1/tmp/s3
> org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Failed to
> register jar :s3. Caught exception.
> at org.apache.pig.PigServer.addJarsFromProperties(PigServer.java:253)
> at org.apache.pig.PigServer.<init>(PigServer.java:239)
> at org.apache.pig.PigServer.<init>(PigServer.java:215)
> at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)
> at org.apache.pig.Main.run(Main.java:492)
> at org.apache.pig.Main.main(Main.java:107)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 101:
> Could not copy s3 to local destination /mnt/var/lib/hadoop/steps/1/tmp/s3
> at
> org.apache.pig.impl.io.FileLocalizer.fetchFile(FileLocalizer.java:765)
> at org.apache.pig.PigServer.registerJar(PigServer.java:483)
> at org.apache.pig.PigServer.addJarsFromProperties(PigServer.java:248)
> ... 10 more
> Caused by: java.io.FileNotFoundException: File does not exist: s3
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:457)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
> at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1256)
> at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1237)
> at
> org.apache.pig.impl.io.FileLocalizer.fetchFile(FileLocalizer.java:763)
> ... 12 more
> ================================================================================
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
