[
https://issues.apache.org/jira/browse/FALCON-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Greene updated FALCON-1787:
--------------------------------
Description:
I have a Pig script that I am using as the workflow for my Falcon process. The
pig script uses HCatalogStorer to write to a HCatalog URI that is the output
feed defined in my Falcon Process Entity. The Pig action in the resulting
Ooozie Workflow generated by Falcon fails with the attached stack trace. The
root is that it is missing a class definitions of
org/apache/hadoop/hive/shims/ShimLoader.
Running the script manually using pig -x tex -useHCatalog <all the -params
passed by Oozie> <path to pig script> results in a successful execution. It's
only once this is called as a Pig activity in the Falcon-generated Oozie
workflow that the missing class definitions manifests.
After some investigation I found that the Oozie workflow.xml is missing a
required sharelib decleration.
>From the workflow.xml generated by Falcon:
<property>
<name>oozie.action.sharelib.for.pig</name>
<value>pig,hcatalog</value>
</property>
If I modify the value to include hive sharelib then the Pig action succeeds and
does not throw a missing class definition error.
Modified workflow.xml property (works):
<property>
<name>oozie.action.sharelib.for.pig</name>
<value>hive,pig,hcatalog</value>
</property>
was:
have a Pig script that I am using as the workflow for my Falcon process. The
pig script uses HCatalogStorer to write to a HCatalog URI that is the output
feed defined in my Falcon Process Entity. The Pig action in the resulting
Ooozie Workflow generated by Falcon fails with the attached stack trace. The
root is that it is missing a class definitions of
org/apache/hadoop/hive/shims/ShimLoader.
Running the script manually using pig -x tex -useHCatalog <all the -params
passed by Oozie> <path to pig script> results in a successful execution. It's
only once this is called as a Pig activity in the Falcon-generated Oozie
workflow that the missing class definitions manifests.
After some investigation I found that the Oozie workflow.xml is missing a
required sharelib decleration.
>From the workflow.xml generated by Falcon:
<property>
<name>oozie.action.sharelib.for.pig</name>
<value>pig,hcatalog</value>
</property>
If I modify the value to include hive sharelib then the Pig action succeeds and
does not throw a missing class definition error.
Modified workflow.xml property (works):
<property>
<name>oozie.action.sharelib.for.pig</name>
<value>hive,pig,hcatalog</value>
</property>
> Ooozie pig-action.xml requires hive sharedlib for HCatalog use
> --------------------------------------------------------------
>
> Key: FALCON-1787
> URL: https://issues.apache.org/jira/browse/FALCON-1787
> Project: Falcon
> Issue Type: Bug
> Components: oozie
> Affects Versions: 0.6.1
> Environment: HDP-2.3.2.0-2950
> Pig 0.15.0.2.3
> Hive 1.2.1.2.3
> Oozie 4.2.0.2.3
> Falcon 0.6.1.2.3
> Reporter: Mark Greene
> Attachments: Stack Trace.txt, Workflow Job Configuration.txt,
> Workflow Pig Action Configuration.txt
>
>
> I have a Pig script that I am using as the workflow for my Falcon process.
> The pig script uses HCatalogStorer to write to a HCatalog URI that is the
> output feed defined in my Falcon Process Entity. The Pig action in the
> resulting Ooozie Workflow generated by Falcon fails with the attached stack
> trace. The root is that it is missing a class definitions of
> org/apache/hadoop/hive/shims/ShimLoader.
> Running the script manually using pig -x tex -useHCatalog <all the -params
> passed by Oozie> <path to pig script> results in a successful execution. It's
> only once this is called as a Pig activity in the Falcon-generated Oozie
> workflow that the missing class definitions manifests.
> After some investigation I found that the Oozie workflow.xml is missing a
> required sharelib decleration.
> From the workflow.xml generated by Falcon:
> <property>
> <name>oozie.action.sharelib.for.pig</name>
> <value>pig,hcatalog</value>
> </property>
> If I modify the value to include hive sharelib then the Pig action succeeds
> and does not throw a missing class definition error.
> Modified workflow.xml property (works):
> <property>
> <name>oozie.action.sharelib.for.pig</name>
> <value>hive,pig,hcatalog</value>
> </property>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
