[
https://issues.apache.org/jira/browse/OOZIE-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029934#comment-16029934
]
Artem Ervits commented on OOZIE-2919:
-------------------------------------
[~rohini] granted I haven't spent too much time researching but I do see JIRA
you contributed https://issues.apache.org/jira/browse/OOZIE-1861
I was following these instructions where tez.lib.uris was specified.
https://community.hortonworks.com/content/supportkb/48935/how-to-launch-oozie-pig-job-using-tez-execution-en.html
I'm able to launch pig action in tez mode now but it gets killed and I wonder
why. I'm getting the following error and granted it's a vendor distribution and
I can follow up on their respective mailing list but perhaps I'm doing
something wrong? I'm happy to change this jira and perhaps contribute a better
example of pig on tez via oozie.
{noformat}
java.security.AccessControlException: User yarn (auth:SIMPLE) cannot perform AM
modify operation
at
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.shutdownSession(DAGClientAMProtocolBlockingPBServerImpl.java:191)
at
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7638)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2345)
{noformat}
my current workflow is below, please let me know what I'm doing wrong?
{noformat}
<workflow-app name="pig-with-tez"
xmlns="uri:oozie:workflow:0.5">
<start to="pig_with_tez"/>
<action name="pig_with_tez">
<pig>
<job-tracker>${resourceManager}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${nameNode}/tmp/output"></delete>
</prepare>
<configuration>
<property>
<name>oozie.action.sharelib.for.pig</name>
<value>pig,hive</value>
</property>
<property>
<name>tez.lib.uris</name>
<value>/hdp/apps/2.6.0.3-8/tez/tez.tar.gz</value>
</property>
</configuration>
<script>/user/admin/oozie/pig-tez/lib/script.pig</script>
<argument>-x</argument>
<argument>tez</argument>
<file>/user/admin/oozie/pig-tez/lib/tez-site.xml</file>
</pig>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>${wf:errorMessage(wf:lastErrorNode())}</message>
</kill>
<end name="end"/>
</workflow-app>
{noformat}
> make Pig run with Tez execution launcher easier
> -----------------------------------------------
>
> Key: OOZIE-2919
> URL: https://issues.apache.org/jira/browse/OOZIE-2919
> Project: Oozie
> Issue Type: Improvement
> Components: core
> Affects Versions: 4.3.0
> Reporter: Artem Ervits
> Assignee: Artem Ervits
>
> I've been trying to make Pig run with Tez via Oozie and was not able to do so
> easily. There's a workaround that requires Tez libs added to oozie sharelib
> rather having its own directory and also requires some properties in the
> oozie wf xml. I propose to have Tez treated as first class citizen as per
> Yahoo talk at Hadoop Summit, they moved off of MR to TEZ for all of their Pig
> workloads.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
