[
https://issues.apache.org/jira/browse/PIG-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-3886:
-------------------------------
Attachment: PIG-3886-2.patch
Here is a new patch that handles s3 paths. [~daijy], can you review this?
There is still a problem \[1\], but I think we can fix it in a separate jira?
(I didn't have time to get it done today.)
\[1\] I have a production job that ships a s3 directory recursively using
{{pig.streaming.cache.files}}, but I couldn't get it working yet. I see files
are shipped to a temporary staging directory on hdfs and are added to task
local resources. But somehow they're not available inside the udf.
> UdfDistributedCache_1 fails in tez branch
> -----------------------------------------
>
> Key: PIG-3886
> URL: https://issues.apache.org/jira/browse/PIG-3886
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.14.0
>
> Attachments: PIG-3886-1.patch, PIG-3886-2.patch
>
>
> The e2e test "UdfDistributedCache_1" fails with the following error-
> {code}
> : Caused by: java.io.FileNotFoundException: ./foodle (No
> such file or directory)
> : at java.io.FileInputStream.open(Native Method)
> : at
> java.io.FileInputStream.<init>(FileInputStream.java:146)
> : at
> java.io.FileInputStream.<init>(FileInputStream.java:101)
> : at java.io.FileReader.<init>(FileReader.java:58)
> : at
> org.apache.pig.test.udf.evalfunc.Udfcachetest.exec(Udfcachetest.java:60)
> : at
> org.apache.pig.test.udf.evalfunc.Udfcachetest.exec(Udfcachetest.java:40)
> : at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:353)
> : ... 14 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
