Shwetha Apologise to trouble everyone but i am new to this falcon. Do i need to check the logs in JobBrowser?
there is stderror Log Length: 188 Error: Could not find or load main class com.jpmc.cib.is.bigdata. falcon.SemsStaticDataProvider Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1] On Thu, Nov 27, 2014 at 11:43 AM, Shwetha GS <[email protected]> wrote: > oozie launches a hadoop job which executes the shell script. Check the > launcher job logs > > On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <[email protected]> > wrote: > > > Srikanth > > > > What do we mean by that in oozie context? > > > > Thanks > > > > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan < > > [email protected]> > > wrote: > > > > > Looks like the script ingest.sh is returning non-zero exit code. > > > > > > Regards > > > Srikanth Sundarrajan > > > > > > > Date: Thu, 27 Nov 2014 11:31:30 +0530 > > > > Subject: Re: Need help for the problem that i faced > > > > From: [email protected] > > > > To: [email protected] > > > > > > > > These are the log > > > > > > > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action > > > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry > state : > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10] > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action > status=DONE > > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] > > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated > in > > > DB! > > > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action > > > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry > > state > > > : > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10] > > > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking > > action, > > > > hadoop job ID [job_1417019189193_0117] status [RUNNING] > > > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action > > > > status=RUNNING > > > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] > > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action > updated > > in > > > > DB! > > > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for > > > action > > > > [0000129-141126162609456-oozie-oozi-W@shell-node] > > > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action > > > completed, > > > > external ID [job_1417019189193_0117] > > > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher > > ERROR, > > > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit > > code > > > > [1] > > > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is > > > considered > > > > as FAILED for SLA > > > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action > > > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state : > > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10] > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE > > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[ > > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] > > APP[shell-wf] > > > > JOB[0000129-141126162609456-oozie-oozi-W] > > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] > > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in > DB! > > > > > > > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <[email protected]> > > > wrote: > > > > > > > > > <exec>ingest.sh</exec> > > > > > Try using absolute path. > > > > > > > > > > None of the logs imply any error. Can you attach all the oozie logs > > > for the > > > > > workflow? > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Hello I am running apache falcon and oozie with shell script to > run > > > the > > > > > > workflow. > > > > > > > > > > > > > > > > > > > > > > > > getting this error > > > > > > > > > > > > *ell-node] Launcher ERROR, reason: Main class > > > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]* > > > > > > *2014-11-26 17:24:06,645 INFO ActionEndXCommand:543 - > > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>] > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf] > > > > > > JOB[0000020-141126162609456-oozie-oozi-W] > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is > > > > > > considered > > > > > > as FAILED for SLA* > > > > > > *2014-11-26 17:24:06,702 INFO ActionStartXCommand:543 - > > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>] > > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf] > > > > > > JOB[0000020-141126162609456-oozie-oozi-W] > > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action > > > > > > [0000020-141126162609456-oozie-* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Workflow* > > > > > > > > > > > > <!-- > > > > > > Licensed to the Apache Software Foundation (ASF) under one > > > > > > or more contributor license agreements. See the NOTICE file > > > > > > distributed with this work for additional information > > > > > > regarding copyright ownership. The ASF licenses this file > > > > > > to you under the Apache License, Version 2.0 (the > > > > > > "License"); you may not use this file except in compliance > > > > > > with the License. You may obtain a copy of the License at > > > > > > > > > > > > http://www.apache.org/licenses/LICENSE-2.0 > > > > > > > > > > > > Unless required by applicable law or agreed to in writing, > > software > > > > > > distributed under the License is distributed on an "AS IS" > BASIS, > > > > > > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > > > > > implied. > > > > > > See the License for the specific language governing permissions > > and > > > > > > limitations under the License. > > > > > > --> > > > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf"> > > > > > > <start to="shell-node"/> > > > > > > <action name="shell-node"> > > > > > > <shell xmlns="uri:oozie:shell-action:0.2"> > > > > > > <job-tracker>${jobTracker}</job-tracker> > > > > > > <name-node>${nameNode}</name-node> > > > > > > <configuration> > > > > > > <property> > > > > > > <name>mapred.job.queue.name</name> > > > > > > <value>${queueName}</value> > > > > > > </property> > > > > > > </configuration> > > > > > > <exec>ingest.sh</exec> > > > > > > <argument>${feedInstancePaths}</argument> > > > > > > <file>${wf:appPath()}/ingest.sh#ingest.sh</file> > > > > > > <!-- <file>/tmp/ingest.sh#ingest.sh</file> --> > > > > > > <!-- <capture-output/> --> > > > > > > </shell> > > > > > > <ok to="end"/> > > > > > > <error to="fail"/> > > > > > > </action> > > > > > > <kill name="fail"> > > > > > > <message>Shell action failed, error > > > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message> > > > > > > </kill> > > > > > > <end name="end"/> > > > > > > </workflow-app> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is the shell script > > > > > > > > > > > > > > > > > > > > > > > > #!/bin/bash > > > > > > > > > > > > > > > > > > java -cp > > > > > > > > > > > > > > > > > "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*" > > > > > > com.bigdata.falcon.StaticDataProvider > > > > > > > > > > > > > > > > > > > > > > > > These are the feed and process and cluster information > > > > > > > > > > > > > > > > > > > > > > > > <feed description="Raw customer email feed" > > name="FINALrawEmailFeed" > > > > > > xmlns="uri:falcon:feed:0.1"> > > > > > > > > > > > > > > > <tags>externalSystem=USWestEmailServers,classification=secure</tags> > > > > > > <groups>churnAnalysisDataPipeline</groups> > > > > > > <frequency>minutes(3)</frequency> > > > > > > <late-arrival cut-off="hours(4)"/> > > > > > > <clusters> > > > > > > <cluster name="SprimaryCluster" type="source"> > > > > > > <validity start="2014-11-21T00:00Z" > > > end="2016-03-31T00:00Z"/> > > > > > > <retention limit="days(90)" action="delete"/> > > > > > > </cluster> > > > > > > </clusters> > > > > > > > > > > > > <locations> > > > > > > <location type="data" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/> > > > > > > <location type="stats" path="/none"/> > > > > > > <location type="meta" path="/none"/> > > > > > > </locations> > > > > > > > > > > > > <ACL owner="ambari-qa" group="users" permission="0777"/> > > > > > > <schema location="/none" provider="none"/> > > > > > > </feed> > > > > > > > > > > > > > > > > > > > > > > > > *Process* > > > > > > > > > > > > *-----------* > > > > > > > > > > > > > > > > > > > > > > > > <?xml version="1.0" encoding="UTF-8"?> > > > > > > <!-- > > > > > > Hourly process, ingests raw emails from all US west email > servers > > > > > > --> > > > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags> > > > > > > > > > > > > > > > > > > <clusters> > > > > > > <cluster name="SprimaryCluster"> > > > > > > <validity start="2014-11-24T09:35Z" > > > end="2014-11-25T10:00Z"/> > > > > > > </cluster> > > > > > > </clusters> > > > > > > > > > > > > <parallel>1</parallel> > > > > > > <order>FIFO</order> > > > > > > <frequency>minutes(3)</frequency> > > > > > > > > > > > > <outputs> > > > > > > <output name="output" feed="FINALrawEmailFeed" > > > > > instance="now(0,0)" > > > > > > /> > > > > > > </outputs> > > > > > > > > > > > > <workflow name="IngestWorkflow" version="2.0.0" > > > > > > engine="oozie" > > path="/user/gaurav123/falcon/demo/apps/ingest/fs" > > > /> > > > > > > > > > > > > <retry policy="periodic" delay="minutes(15)" attempts="3" /> > > > > > > > > > > > > </process> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you please let me know what is the issue , I have tried since > > > long > > > > > two > > > > > > days? but unable to run the workflow with falcon process > > > > > > > > > > > > Also got this exception > > > > > > > > > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 - > > > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-] > > > > > > JOB[0000099-141126162609456-oozie-oozi-W] > > > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action] > > XException, > > > > > > org.apache.oozie.command.CommandException: E0800: Action it is > not > > > > > > running its in [PREP] state, action > > > > > > [0000099-141126162609456-oozie-oozi-W@user-action] > > > > > > > > > > > > > > > > -- > > > > > _____________________________________________________________ > > > > > The information contained in this communication is intended solely > > for > > > the > > > > > use of the individual or entity to whom it is addressed and others > > > > > authorized to receive it. It may contain confidential or legally > > > privileged > > > > > information. If you are not the intended recipient you are hereby > > > notified > > > > > that any disclosure, copying, distribution or taking any action in > > > reliance > > > > > on the contents of this information is strictly prohibited and may > be > > > > > unlawful. If you have received this communication in error, please > > > notify > > > > > us immediately by responding to this email and then delete it from > > your > > > > > system. The firm is neither liable for the proper and complete > > > transmission > > > > > of the information contained in this communication nor for any > delay > > > in its > > > > > receipt. > > > > > > > > > > > > > > > -- > _____________________________________________________________ > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its > receipt. >
