thanks issue is resolved.
On 11/27/14, Shwetha GS <[email protected]> wrote: > Try the java action in oozie: > http://oozie.apache.org/docs/4.0.1/WorkflowFunctionalSpec.html#a3.2.7_Java_Action > > On Thu, Nov 27, 2014 at 11:49 AM, Gaurav Agarwal <[email protected]> > wrote: > >> Shwetha >> >> Apologise to trouble everyone but i am new to this falcon. Do i need to >> check the logs in JobBrowser? >> >> there is stderror >> >> Log Length: 188 >> Error: Could not find or load main class com.jpmc.cib.is.bigdata. >> falcon.SemsStaticDataProvider >> Failing Oozie Launcher, Main class >> [org.apache.oozie.action.hadoop.ShellMain], >> exit code [1] >> >> >> >> >> >> On Thu, Nov 27, 2014 at 11:43 AM, Shwetha GS <[email protected]> >> wrote: >> >> > oozie launches a hadoop job which executes the shell script. Check the >> > launcher job logs >> > >> > On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <[email protected]> >> > wrote: >> > >> > > Srikanth >> > > >> > > What do we mean by that in oozie context? >> > > >> > > Thanks >> > > >> > > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan < >> > > [email protected]> >> > > wrote: >> > > >> > > > Looks like the script ingest.sh is returning non-zero exit code. >> > > > >> > > > Regards >> > > > Srikanth Sundarrajan >> > > > >> > > > > Date: Thu, 27 Nov 2014 11:31:30 +0530 >> > > > > Subject: Re: Need help for the problem that i faced >> > > > > From: [email protected] >> > > > > To: [email protected] >> > > > > >> > > > > These are the log >> > > > > >> > > > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action >> > > > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry >> > state : >> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10] >> > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] >> > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action >> > status=DONE >> > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] >> > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action >> > > > > updated >> > in >> > > > DB! >> > > > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start >> action >> > > > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry >> > > state >> > > > : >> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10] >> > > > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking >> > > action, >> > > > > hadoop job ID [job_1417019189193_0117] status [RUNNING] >> > > > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] >> > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action >> > > > > status=RUNNING >> > > > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] >> > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action >> > updated >> > > in >> > > > > DB! >> > > > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback >> for >> > > > action >> > > > > [0000129-141126162609456-oozie-oozi-W@shell-node] >> > > > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action >> > > > completed, >> > > > > external ID [job_1417019189193_0117] >> > > > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher >> > > ERROR, >> > > > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], >> > > > > exit >> > > code >> > > > > [1] >> > > > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is >> > > > considered >> > > > > as FAILED for SLA >> > > > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action >> > > > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state >> : >> > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10] >> > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] >> > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action >> status=DONE >> > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[ >> > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] >> > > APP[shell-wf] >> > > > > JOB[0000129-141126162609456-oozie-oozi-W] >> > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] >> > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated >> > > > > in >> > DB! >> > > > > >> > > > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS < >> [email protected]> >> > > > wrote: >> > > > > >> > > > > > <exec>ingest.sh</exec> >> > > > > > Try using absolute path. >> > > > > > >> > > > > > None of the logs imply any error. Can you attach all the oozie >> logs >> > > > for the >> > > > > > workflow? >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal < >> > [email protected] >> > > > >> > > > > > wrote: >> > > > > > >> > > > > > > Hello I am running apache falcon and oozie with shell script >> > > > > > > to >> > run >> > > > the >> > > > > > > workflow. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > getting this error >> > > > > > > >> > > > > > > *ell-node] Launcher ERROR, reason: Main class >> > > > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]* >> > > > > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 - >> > > > > > > SERVER[sandbox.hortonworks.com >> > > > > > > <http://sandbox.hortonworks.com >> >] >> > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf] >> > > > > > > JOB[0000020-141126162609456-oozie-oozi-W] >> > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR >> is >> > > > > > > considered >> > > > > > > as FAILED for SLA* >> > > > > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 - >> > > > > > > SERVER[sandbox.hortonworks.com >> > > > > > > <http://sandbox.hortonworks.com >> >] >> > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf] >> > > > > > > JOB[0000020-141126162609456-oozie-oozi-W] >> > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start >> > > > > > > action >> > > > > > > [0000020-141126162609456-oozie-* >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > *Workflow* >> > > > > > > >> > > > > > > <!-- >> > > > > > > Licensed to the Apache Software Foundation (ASF) under one >> > > > > > > or more contributor license agreements. See the NOTICE >> > > > > > > file >> > > > > > > distributed with this work for additional information >> > > > > > > regarding copyright ownership. The ASF licenses this file >> > > > > > > to you under the Apache License, Version 2.0 (the >> > > > > > > "License"); you may not use this file except in compliance >> > > > > > > with the License. You may obtain a copy of the License at >> > > > > > > >> > > > > > > http://www.apache.org/licenses/LICENSE-2.0 >> > > > > > > >> > > > > > > Unless required by applicable law or agreed to in writing, >> > > software >> > > > > > > distributed under the License is distributed on an "AS IS" >> > BASIS, >> > > > > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either >> > > > > > > express >> or >> > > > > > implied. >> > > > > > > See the License for the specific language governing >> permissions >> > > and >> > > > > > > limitations under the License. >> > > > > > > --> >> > > > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf"> >> > > > > > > <start to="shell-node"/> >> > > > > > > <action name="shell-node"> >> > > > > > > <shell xmlns="uri:oozie:shell-action:0.2"> >> > > > > > > <job-tracker>${jobTracker}</job-tracker> >> > > > > > > <name-node>${nameNode}</name-node> >> > > > > > > <configuration> >> > > > > > > <property> >> > > > > > > <name>mapred.job.queue.name</name> >> > > > > > > <value>${queueName}</value> >> > > > > > > </property> >> > > > > > > </configuration> >> > > > > > > <exec>ingest.sh</exec> >> > > > > > > <argument>${feedInstancePaths}</argument> >> > > > > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file> >> > > > > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> --> >> > > > > > > <!-- <capture-output/> --> >> > > > > > > </shell> >> > > > > > > <ok to="end"/> >> > > > > > > <error to="fail"/> >> > > > > > > </action> >> > > > > > > <kill name="fail"> >> > > > > > > <message>Shell action failed, error >> > > > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message> >> > > > > > > </kill> >> > > > > > > <end name="end"/> >> > > > > > > </workflow-app> >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > This is the shell script >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > #!/bin/bash >> > > > > > > >> > > > > > > >> > > > > > > java -cp >> > > > > > > >> > > > > > >> > > > >> > > >> > >> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*" >> > > > > > > com.bigdata.falcon.StaticDataProvider >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > These are the feed and process and cluster information >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > <feed description="Raw customer email feed" >> > > name="FINALrawEmailFeed" >> > > > > > > xmlns="uri:falcon:feed:0.1"> >> > > > > > > >> > > > > > > >> > > > >> > > > <tags>externalSystem=USWestEmailServers,classification=secure</tags> >> > > > > > > <groups>churnAnalysisDataPipeline</groups> >> > > > > > > <frequency>minutes(3)</frequency> >> > > > > > > <late-arrival cut-off="hours(4)"/> >> > > > > > > <clusters> >> > > > > > > <cluster name="SprimaryCluster" type="source"> >> > > > > > > <validity start="2014-11-21T00:00Z" >> > > > end="2016-03-31T00:00Z"/> >> > > > > > > <retention limit="days(90)" action="delete"/> >> > > > > > > </cluster> >> > > > > > > </clusters> >> > > > > > > >> > > > > > > <locations> >> > > > > > > <location type="data" >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > >> > > >> > >> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/> >> > > > > > > <location type="stats" path="/none"/> >> > > > > > > <location type="meta" path="/none"/> >> > > > > > > </locations> >> > > > > > > >> > > > > > > <ACL owner="ambari-qa" group="users" permission="0777"/> >> > > > > > > <schema location="/none" provider="none"/> >> > > > > > > </feed> >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > *Process* >> > > > > > > >> > > > > > > *-----------* >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > <?xml version="1.0" encoding="UTF-8"?> >> > > > > > > <!-- >> > > > > > > Hourly process, ingests raw emails from all US west email >> > servers >> > > > > > > --> >> > > > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1"> >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > >> > > >> > >> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags> >> > > > > > > >> > > > > > > >> > > > > > > <clusters> >> > > > > > > <cluster name="SprimaryCluster"> >> > > > > > > <validity start="2014-11-24T09:35Z" >> > > > end="2014-11-25T10:00Z"/> >> > > > > > > </cluster> >> > > > > > > </clusters> >> > > > > > > >> > > > > > > <parallel>1</parallel> >> > > > > > > <order>FIFO</order> >> > > > > > > <frequency>minutes(3)</frequency> >> > > > > > > >> > > > > > > <outputs> >> > > > > > > <output name="output" feed="FINALrawEmailFeed" >> > > > > > instance="now(0,0)" >> > > > > > > /> >> > > > > > > </outputs> >> > > > > > > >> > > > > > > <workflow name="IngestWorkflow" version="2.0.0" >> > > > > > > engine="oozie" >> > > path="/user/gaurav123/falcon/demo/apps/ingest/fs" >> > > > /> >> > > > > > > >> > > > > > > <retry policy="periodic" delay="minutes(15)" attempts="3" >> /> >> > > > > > > >> > > > > > > </process> >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > Can you please let me know what is the issue , I have tried >> since >> > > > long >> > > > > > two >> > > > > > > days? but unable to run the workflow with falcon process >> > > > > > > >> > > > > > > Also got this exception >> > > > > > > >> > > > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 - >> > > > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] >> APP[-] >> > > > > > > JOB[0000099-141126162609456-oozie-oozi-W] >> > > > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action] >> > > XException, >> > > > > > > org.apache.oozie.command.CommandException: E0800: Action it >> > > > > > > is >> > not >> > > > > > > running its in [PREP] state, action >> > > > > > > [0000099-141126162609456-oozie-oozi-W@user-action] >> > > > > > > >> > > > > > >> > > > > > -- >> > > > > > _____________________________________________________________ >> > > > > > The information contained in this communication is intended >> solely >> > > for >> > > > the >> > > > > > use of the individual or entity to whom it is addressed and >> others >> > > > > > authorized to receive it. It may contain confidential or >> > > > > > legally >> > > > privileged >> > > > > > information. If you are not the intended recipient you are >> > > > > > hereby >> > > > notified >> > > > > > that any disclosure, copying, distribution or taking any action >> in >> > > > reliance >> > > > > > on the contents of this information is strictly prohibited and >> may >> > be >> > > > > > unlawful. If you have received this communication in error, >> please >> > > > notify >> > > > > > us immediately by responding to this email and then delete it >> from >> > > your >> > > > > > system. The firm is neither liable for the proper and complete >> > > > transmission >> > > > > > of the information contained in this communication nor for any >> > delay >> > > > in its >> > > > > > receipt. >> > > > > > >> > > > >> > > > >> > > >> > >> > -- >> > _____________________________________________________________ >> > The information contained in this communication is intended solely for >> the >> > use of the individual or entity to whom it is addressed and others >> > authorized to receive it. It may contain confidential or legally >> privileged >> > information. If you are not the intended recipient you are hereby >> notified >> > that any disclosure, copying, distribution or taking any action in >> reliance >> > on the contents of this information is strictly prohibited and may be >> > unlawful. If you have received this communication in error, please >> > notify >> > us immediately by responding to this email and then delete it from your >> > system. The firm is neither liable for the proper and complete >> transmission >> > of the information contained in this communication nor for any delay in >> its >> > receipt. >> > >> > > -- > _____________________________________________________________ > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > > of the information contained in this communication nor for any delay in its > > receipt. >
