[ https://issues.apache.org/jira/browse/PIG-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15907866#comment-15907866 ]
Nandor Kollar commented on PIG-5187: ------------------------------------ When no files are shipped/cached, cacheFiles and shipFiles still wants to add a file with empty string as name. Added PIG-5187_1.patch to fix this problem. > UdfDistributedCache_1 is failing with spark exec type > ----------------------------------------------------- > > Key: PIG-5187 > URL: https://issues.apache.org/jira/browse/PIG-5187 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: Nandor Kollar > Assignee: Nandor Kollar > Fix For: spark-branch > > Attachments: PIG-5187_1.patch, PIG-5187.patch > > > UdfDistributedCache_1 fails, because Java EvalFunc#getCacheFiles is not > handled, thus files are not added to SparkContext: > {code} > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2078: > Caught error from UDF: org.apache.pig.test.udf.evalfunc.Udfcachetest > [./foodle (No such file or directory)] > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:358) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:430) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:341) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:406) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:323) > at > org.apache.pig.backend.hadoop.executionengine.spark.converter.ForEachConverter$ForEachFunction$1$1.getNextResult(ForEachConverter.java:109) > at > org.apache.pig.backend.hadoop.executionengine.spark.converter.OutputConsumerIterator.readNext(OutputConsumerIterator.java:69) > ... 15 more > Caused by: java.io.FileNotFoundException: ./foodle (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:146) > at java.io.FileInputStream.<init>(FileInputStream.java:101) > at java.io.FileReader.<init>(FileReader.java:58) > at > org.apache.pig.test.udf.evalfunc.Udfcachetest.exec(Udfcachetest.java:60) > at > org.apache.pig.test.udf.evalfunc.Udfcachetest.exec(Udfcachetest.java:40) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330) > {code} > Similarly, EvalFunc#getShipFiles is not handled. -- This message was sent by Atlassian JIRA (v6.3.15#6346)