[
https://issues.apache.org/jira/browse/PIG-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015789#comment-14015789
]
Cheolsoo Park commented on PIG-3478:
------------------------------------
[~lbendig], unfortunately, I am getting the following error while running the
e2e tests-
{code}
Caused by: java.io.IOException: Could not create directory:
/apps/hadoop/2.4.0/logs/udfOutput
at
org.apache.pig.scripting.ScriptingOutputCapturer.getTaskLogDir(ScriptingOutputCapturer.java:103)
at
org.apache.pig.scripting.ScriptingOutputCapturer.getStandardOutputRootWriteLocation(ScriptingOutputCapturer.java:85)
at
org.apache.pig.impl.builtin.StreamingUDF.constructCommand(StreamingUDF.java:186)
at
org.apache.pig.impl.builtin.StreamingUDF.startUdfController(StreamingUDF.java:163)
at
org.apache.pig.impl.builtin.StreamingUDF.initialize(StreamingUDF.java:156)
at org.apache.pig.impl.builtin.StreamingUDF.exec(StreamingUDF.java:146)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:344)
{code}
The problem is that users don't always have write permissions to hadoop.log.dir
{{/apps/hadoop/2.4.0/logs}}. Shouldn't we write to the temp dir instead?
> Make StreamingUDF work for Hadoop 2
> -----------------------------------
>
> Key: PIG-3478
> URL: https://issues.apache.org/jira/browse/PIG-3478
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Jeremy Karn
> Fix For: 0.14.0
>
> Attachments: PIG-3478-2.patch, PIG-3478.patch
>
>
> PIG-2417 introduced Streaming UDF. However, it does not work under Hadoop 2.
> Both unit tests/e2e tests under Haodop 2 fails. We need to fix it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
