[ https://issues.apache.org/jira/browse/PIG-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xuefu Zhang updated PIG-4746: ----------------------------- Summary: Ensure spark can be run as PIG action in Oozie (was: Ensure spork can be run as PIG action in Oozie) > Ensure spark can be run as PIG action in Oozie > ---------------------------------------------- > > Key: PIG-4746 > URL: https://issues.apache.org/jira/browse/PIG-4746 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: Pallavi Rao > Assignee: Srikanth Sundarrajan > Fix For: spark-branch > > > I was able get PIG on SPARK going with Oozie. But, only in "local" mode. Here > is what I did: > 1. Used workflow schema version uri:oozie:workflow:0.2 and passed exectype as > an argument. > 2. Copied Spark jars + kyro jar into workflow app lib. > To get spork going in yarn-client mode, couple of enhancements will need to > be made: > 1. Right now, spark launcher reads SPARK_MASTER as env. variable. Need to > make this a PIG property. > 2. The spark libraries need to be in classpath of the driver in case of > yarn-clientmode. This will need a fix similar to PIG-4667 -- This message was sent by Atlassian JIRA (v6.3.4#6332)