Thanks Pallavi, i still had the issue, will look into the jira and patch up.
Sent from my iPhone > On Aug 19, 2015, at 11:53 PM, Pallavi Rao <[email protected]> wrote: > > Siddharth, > Not sure if you were able to resolve the issue, we realized there was issue > with EL expression resolution on Apache Oozie installation. Hence, the issue > you noticed. Here is the JIRA with details and patch. > > Hope this helps, > Pallavi > >> On Thu, Aug 13, 2015 at 9:12 AM, Pallavi Rao <[email protected]> wrote: >> Siddharth, >> The only reason that error could occur is when the EL extension of Falcon is >> not getting picked up by Oozie. Can you double check the following: >> 1. oozie-site.xml has the extensions. You must have the following lines: >> dataIn=org.apache.oozie.extensions.OozieELExtensions#ph1_dataIn_echo and >> >> dataIn=org.apache.oozie.extensions.OozieELExtensions#ph3_dataIn, >> >> 2. You must have the following file in the oozie installation path : >> >> oozie-server/webapps/oozie/WEB-INF/lib/falcon-oozie-el-extension-0.7-SNAPSHOT.jar. >> It must have read all permission. >> >> 3. Hope you have restarted Oozie. >> >> If you are seeing errors even after ensuring the above. Please mail across >> the Oozie logs, please. >> >> Regards, >> >> Pallavi >> >> >>> On Sun, Aug 9, 2015 at 10:52 AM, Siddharth Tiwari >>> <[email protected]> wrote: >>> Hi Pallavi >>> I made the changes you suggested, I am still getting the following error >>> >>> Caused by: org.apache.falcon.FalconException: E1004 : E1004: Expression >>> language evaluation error, Unable to evaluate :${dataIn('input', 'null')}: >>> at >>> org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:223) >>> at >>> org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:171) >>> at >>> org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:93) >>> ... 60 more >>> Caused by: E1004 : E1004: Expression language evaluation error, Unable to >>> evaluate :${dataIn('input', 'null')}: >>> at >>> org.apache.oozie.client.OozieClient.handleError(OozieClient.java:542) >>> at >>> org.apache.oozie.client.OozieClient$JobSubmit.call(OozieClient.java:625) >>> at >>> org.apache.oozie.client.OozieClient$JobSubmit.call(OozieClient.java:595) >>> at >>> org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:514) >>> at org.apache.oozie.client.OozieClient.dryrun(OozieClient.java:735) >>> at >>> org.apache.oozie.client.ProxyOozieClient.access$401(ProxyOozieClient.java:48) >>> at >>> org.apache.oozie.client.ProxyOozieClient$4.call(ProxyOozieClient.java:177) >>> at >>> org.apache.oozie.client.ProxyOozieClient$4.call(ProxyOozieClient.java:174) >>> at org.apache.oozie.client.OozieClient.doAs(OozieClient.java:198) >>> at >>> org.apache.oozie.client.ProxyOozieClient.dryrun(ProxyOozieClient.java:174) >>> at >>> org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:221) >>> >>> The process I am trying to schedule is the one below:- >>> >>> >>> <?xml version="1.0" encoding="UTF-8"?> >>> <!-- >>> Hourly process, cleanses raw data >>> --> >>> <process name="cleanseEmailProcess" xmlns="uri:falcon:process:0.1"> >>> >>> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup</tags> >>> >>> >>> <clusters> >>> <cluster name="primaryCluster"> >>> <validity start="2014-02-28T00:00Z" end="2016-03-31T00:00Z"/> >>> </cluster> >>> </clusters> >>> >>> <parallel>1</parallel> >>> <order>FIFO</order> >>> <frequency>hours(1)</frequency> >>> >>> <inputs> >>> <input name="input" feed="rawEmailFeed" start="now(0,0)" >>> end="now(0,0)" /> >>> </inputs> >>> >>> <outputs> >>> <output name="output" feed="cleansedEmailFeed" instance="now(0,0)" >>> /> >>> </outputs> >>> >>> <workflow name="emailCleanseWorkflow" version="5.0" >>> engine="pig" path="/user/ambari-qa/falcon/demo/apps/pig/id.pig" /> >>> >>> <retry policy="periodic" delay="minutes(15)" attempts="3" /> >>> <!-- <late-process policy="exp-backoff" delay="hours(1)"> >>> <late-input input="input" workflow-path="/apps/clickstream/late" /> >>> </late-process> --> >>> >>> </process> >>> >>> >>> >>> >>> *------------------------* >>> Cheers !!! >>> Siddharth Tiwari >>> Have a refreshing day !!! >>> "Every duty is holy, and devotion to duty is the highest form of worship of >>> God.” >>> "Maybe other people will try to limit me but I don't limit myself" >>> >>> >>> > Date: Fri, 7 Aug 2015 14:03:50 +0530 >>> > Subject: Re: Installation procedure for Faclon for Apache hadoop >>> > From: [email protected] >>> > To: [email protected] >>> > CC: [email protected] >>> > >>> > Hey Siddharth, >>> > If you are not using Oozie distro that is packaged during falcon build, >>> > ensure the following is Oozie changes are done before you start Falcon >>> > server. >>> > >>> > 1. >>> > >>> > Copy <falcon base >>> > dir>/oozie/libext/falcon-oozie-el-extension-0.7-incubating-SNAPSHOT.jar >>> > to >>> > <oozie base dir>/libext. >>> > 2. >>> > >>> > Modify oozie site xml. Copy all EL related properties from <falcon >>> > install dir>oozie/conf/oozie-site.xml to <oozie base >>> > dir>/conf/oozie-site.xml >>> > 3. >>> > >>> > Restart oozie : >>> > >>> > >>> > - >>> > >>> > sudo -u oozie ./bin/oozie-start.sh >>> > - >>> > >>> > sudo -u oozie bin/oozie-setup.sh prepare-war >>> > - >>> > >>> > sudo -u oozie ./bin/oozie-start.sh >>> > >>> > Just realized we have missed this information in our installation docs. >>> > Have opened JIRA <https://issues.apache.org/jira/browse/FALCON-1378> for >>> >>> > the same. >>> > >>> > I assume that you have information on how to setup up a standalone Oozie >>> > instance. If not, let me know, I can pass along the same to you. >>> > >>> > Regards, >>> > Pallavi >>> > >>> > On Fri, Aug 7, 2015 at 1:33 PM, Siddharth Tiwari >>> > <[email protected]> >>> > wrote: >>> > >>> > > Hi Srikanth >>> > > I am able to build it successfully in embedded mode, the issue is >>> > > primarily interacting with oozie and setting up oozie to interact with >>> > > Falcon, are there any particular steps to setup oozie for Falcon ? >>> > > Secondly >>> > > is the version packed with HDP different from the one available in the >>> > > git >>> > > ? Because I cannot see the xml forms setup on the gui. >>> > > >>> > > Sent from my iPhone >>> > > >>> > > > On Aug 7, 2015, at 12:41 AM, Srikanth Sundarrajan >>> > > > <[email protected]> >>> > > wrote: >>> > > > >>> > > > HI Siddharth, >>> > > > It should be fairly straight forward to use Falcon with hadoop 2.5.0 >>> > > > and >>> > > above. Standard installation instructions should suffice. Is there a >>> > > particular step where you are stuck in getting it setup ? >>> > > > >>> > > > Regards >>> > > > Srikanth Sundarrajan >>> > > > >>> > > >> From: [email protected] >>> > > >> To: [email protected] >>> > > >> Subject: Installation procedure for Faclon for Apache hadoop >>> > > >> Date: Fri, 7 Aug 2015 07:13:52 +0000 >>> > > >> >>> > > >> Hi Team,is there any documentation available for installing and >>> > > configuring falcon for stock vanila Apache hadoop ? It fairly easier fro >>> > > HDP using Ambari, but the procedure is not very clear for installing it >>> > > with Stock Apache hadoop. Any help in this regard will be highly >>> > > appreciated. Thank you in advanced. >>> > > >> >>> > > >> *------------------------* >>> > > >> >>> > > >> Cheers !!! >>> > > >> >>> > > >> Siddharth Tiwari >>> > > >> >>> > > >> Have a refreshing day !!! >>> > > >> "Every duty is holy, and devotion to duty is the highest form of >>> > > worship of God.” >>> > > >> >>> > > >> "Maybe other people will try to limit me but I don't limit myself" >>> > > > >>> > > >>> > >>> > -- >>> > _____________________________________________________________ >>> > The information contained in this communication is intended solely for >>> > the >>> > use of the individual or entity to whom it is addressed and others >>> > authorized to receive it. It may contain confidential or legally >>> > privileged >>> > information. If you are not the intended recipient you are hereby >>> > notified >>> > that any disclosure, copying, distribution or taking any action in >>> > reliance >>> > on the contents of this information is strictly prohibited and may be >>> > unlawful. If you have received this communication in error, please notify >>> > us immediately by responding to this email and then delete it from your >>> > system. The firm is neither liable for the proper and complete >>> > transmission >>> > of the information contained in this communication nor for any delay in >>> > its >>> > receipt. > > > _____________________________________________________________ > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others authorized > to receive it. It may contain confidential or legally privileged information. > If you are not the intended recipient you are hereby notified that any > disclosure, copying, distribution or taking any action in reliance on the > contents of this information is strictly prohibited and may be unlawful. If > you have received this communication in error, please notify us immediately > by responding to this email and then delete it from your system. The firm is > neither liable for the proper and complete transmission of the information > contained in this communication nor for any delay in its receipt.
