[
https://issues.apache.org/jira/browse/OOZIE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17228611#comment-17228611
]
Junfan Zhang commented on OOZIE-3581:
-------------------------------------
Oozie uses two methods to detect the status of tasks, active and passive.
Checkerservice is used to detect the status of the task. But callback will
notify Oozie, and then trigger the detection. So if there is a delay, it means
that the queue is backlogged.
> Callback does not applied in Oozie server, workflows stuk in RUNNING states.
> ----------------------------------------------------------------------------
>
> Key: OOZIE-3581
> URL: https://issues.apache.org/jira/browse/OOZIE-3581
> Project: Oozie
> Issue Type: Bug
> Components: action, workflow
> Affects Versions: 4.3.1
> Reporter: Kotsubinsky Victor
> Priority: Critical
>
> oozie version 4.3.1.3.1.0.0-78
> with HDP3.10 stack , release provides Oozie 4.3.1 and the additional Apache
> patches listed here:
> [https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/release-notes/content/patch_oozie.html]
> I use Hadoop kerberized cluster, run on OOZIE, YARN Mr jobs.
> 1. OOzie run mr-job via YARN
> 2. After YARN mr job completed,YYARN mr-job Successfully sent Callback
> request to OOzie,
> 3. in logs OOzie server, i can see this request, but OOZIE does not apply
> this callback request, so in WF-action-id i still see RUNNING state (until
> action.check process check wf-ids and swith action-id to SUCCESS state)
> LOGS in YARN:
> 2020-01-27 12:16:39,749 INFO [Thread-78] org.eclipse.jetty.util.log: Job end
> notification trying
> http://hdp3-oozie:11000/oozie/callback?id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
> 2020-01-27 12:16:39,772 INFO [Thread-78] org.eclipse.jetty.util.log: Job end
> notification to
> http://hdp3-oozie:11000/oozie/callback?id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
> succeeded
> 2020-01-27 12:16:39,772 INFO [Thread-78] org.eclipse.jetty.util.log: Job end
> notification succeeded for job_1579778851579_31505
>
> Oozie logs about this event:
> 2020-01-27 12:16:39,770 DEBUG CallbackServlet:526 - SERVER[hdp3-oo-2] USER[-]
> GROUP[-] TOKEN[-] APP[-] JOB[0005607-200123121357414-oozie-oozi-W]
> ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java]
> Received a CallbackServlet.doGet() with query string
> id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
> 2020-01-27 12:16:39,776 DEBUG CompletedActionXCommand:526 - SERVER[hdp3-oo-2]
> USER[-] GROUP[-] TOKEN[] APP[-] JOB[0005607-200123121357414-oozie-oozi-W]
> ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java]
> Execute command [callback] key [null]
> 2020-01-27 12:16:39,776 DEBUG CompletedActionXCommand:526 - SERVER[hdp3-oo-2]
> USER[-] GROUP[-] TOKEN[] APP[-] JOB[0005607-200123121357414-oozie-oozi-W]
> ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java]
> Queuing [1] commands with delay [0]ms
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
