Added: oozie/site/trunk/content/resources/docs/5.2.1/DG_CoordinatorRerun.html URL: http://svn.apache.org/viewvc/oozie/site/trunk/content/resources/docs/5.2.1/DG_CoordinatorRerun.html?rev=1886952&view=auto ============================================================================== --- oozie/site/trunk/content/resources/docs/5.2.1/DG_CoordinatorRerun.html (added) +++ oozie/site/trunk/content/resources/docs/5.2.1/DG_CoordinatorRerun.html Fri Feb 26 14:14:19 2021 @@ -0,0 +1,177 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2021-02-26 + | Rendered using Apache Maven Fluido Skin 1.4 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20210226" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Oozie – </title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> + + + </head> + <body class="topBarDisabled"> + + + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="https://oozie.apache.org/"; id="bannerLeft"> + <img src="https://oozie.apache.org/images/oozie_200x.png"; alt="Oozie"/> + </a> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="http://www.apache.org/"; class="externalLink" title="Apache"> + Apache</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../../" title="Oozie"> + Oozie</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../" title="docs"> + docs</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="./" title="5.2.1"> + 5.2.1</a> + <span class="divider">/</span> + </li> + <li class="active "></li> + + + + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2021-02-26</li> + <li id="projectVersion" class="pull-right"> + Version: 5.2.1 + </li> + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span2"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + </ul> + + + + <hr /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span10" > + + <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p> +<h1>Coordinator Rerun</h1> +<ul> +<li><a href="#Pre-Conditions">Pre-Conditions</a></li> +<li><a href="#Rerun_Arguments">Rerun Arguments</a></li> +<li><a href="#Rerun_coordinator_actions">Rerun coordinator actions</a></li></ul> + +<div class="section"> +<h2><a name="Pre-Conditions"></a>Pre-Conditions</h2> +<ul> + +<li>Rerun coordinator action must be in TIMEDOUT/SUCCEEDED/KILLED/FAILED.</li> +<li>Coordinator actions cannot be rerun if the coordinator job is in the PREP or IGNORED state.</li> +<li>Rerun a PAUSED coordinator job, the status and pause time and pending flag will not be reset.</li> +<li>Rerun a SUSPENDED coordinator job, the status will reset to RUNNING.</li> +<li>All rerun actions must exist already.</li> +<li>Coordinator Rerun will only use the original configs from first run.</li> +<li>Coordinator Rerun will not re-read the coordinator.xml in hdfs.</li> +</ul></div> +<div class="section"> +<h2><a name="Rerun_Arguments"></a>Rerun Arguments</h2> + +<div> +<div> +<pre class="source">$oozie job -rerun <coord_Job_id> [-nocleanup] [-refresh] [-failed] [-config <arg>] +[-action 1, 3-4, 7-40] (-action or -date is required to rerun.) +[-date 2009-01-01T01:00Z::2009-05-31T23:59Z, 2009-11-10T01:00Z, 2009-12-31T22:00Z] +(if neither -action nor -date is given, the exception will be thrown.) +</pre></div></div> + +<ul> + +<li>Either -action or -date should be given.</li> +<li>If -action and -date both are given, an error will be thrown.</li> +<li>Multiple ranges can be used in -action or -date. See the above examples.</li> +<li>If one of action in the given list of -action does not exist or not in terminal state, the rerun throws an error.</li> +<li>The dates specified in -date must be UTC.</li> +<li>Single date specified in -date must be able to find an action with matched nominal time to be effective.</li> +<li>If -nocleanup is given, coordinator directories will not be removed; otherwise the ‘output-event’ will be deleted, unless nocleanup attribute is explicitly set in coordinator.xml</li> +<li>If -refresh is set, new dataset is re-evaluated for latest() and future().</li> +<li>If -refresh is set, all dependencies will be re-checked; otherwise only missed dependencies will be checked.</li> +<li>If -failed is set, re-runs the failed workflow actions of the coordinator actions.</li> +<li>-config can be used to supply properties to workflow by job configuration file ‘.xml’ or ‘.properties’.</li> +</ul></div> +<div class="section"> +<h2><a name="Rerun_coordinator_actions"></a>Rerun coordinator actions</h2> +<ul> + +<li>Rerun terminated (timeout, succeeded, killed, failed) coordinator actions.</li> +<li>By default, Oozie will delete the ‘output-event’ directories before changing actions’ status and materializing actions.</li> +<li>If coordinator job is RUNNING, rerun actions will be materialized and compete with current running actions.</li> +<li>Rerun for job, user should use job’s start date and end date in -date.</li> +<li>If the user specifies a date range (say Jan 1 to May 1), the actions that will be re-run are the existing actions within that range. If the existing actions are action #5….#40, which map to Jan 15 to Feb 15, then only those actions will run.</li> +<li>The rerun action_id and nominal_time of the actions which are eligible to rerun will be returned.</li> +</ul> +<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row-fluid"> + <p >Copyright © 2021 + <a href="https://www.apache.org";>Apache Software Foundation</a>. + All rights reserved. + + </p> + </div> + + + </div> + </footer> + </body> +</html>
Added: oozie/site/trunk/content/resources/docs/5.2.1/DG_CustomActionExecutor.html URL: http://svn.apache.org/viewvc/oozie/site/trunk/content/resources/docs/5.2.1/DG_CustomActionExecutor.html?rev=1886952&view=auto ============================================================================== --- oozie/site/trunk/content/resources/docs/5.2.1/DG_CustomActionExecutor.html (added) +++ oozie/site/trunk/content/resources/docs/5.2.1/DG_CustomActionExecutor.html Fri Feb 26 14:14:19 2021 @@ -0,0 +1,210 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2021-02-26 + | Rendered using Apache Maven Fluido Skin 1.4 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20210226" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Oozie – </title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> + + + </head> + <body class="topBarDisabled"> + + + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="https://oozie.apache.org/"; id="bannerLeft"> + <img src="https://oozie.apache.org/images/oozie_200x.png"; alt="Oozie"/> + </a> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="http://www.apache.org/"; class="externalLink" title="Apache"> + Apache</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../../" title="Oozie"> + Oozie</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../" title="docs"> + docs</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="./" title="5.2.1"> + 5.2.1</a> + <span class="divider">/</span> + </li> + <li class="active "></li> + + + + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2021-02-26</li> + <li id="projectVersion" class="pull-right"> + Version: 5.2.1 + </li> + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span2"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + </ul> + + + + <hr /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span10" > + + <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p> +<h1>Custom Action Nodes</h1> +<ul> +<li><a href="#Introduction">Introduction</a></li> +<li><a href="#Writing_a_custom_Action_Node">Writing a custom Action Node</a> +<ul> +<li><a href="#ActionExecutor.Context">ActionExecutor.Context</a></li> +<li><a href="#Constructor">Constructor</a></li> +<li><a href="#initActionType">initActionType()</a></li> +<li><a href="#startActionExecutor.Context_context_Action_action">start(ActionExecutor.Context context, Action action)</a></li> +<li><a href="#checkActionExecutor.Context_context_Action_action">check(ActionExecutor.Context context, Action action)</a></li> +<li><a href="#killActionExecutor.Context_context_Action_action">kill(ActionExecutor.Context context, Action action)</a></li> +<li><a href="#endActionExecutor.Context_context_Action_action">end(ActionExecutor.Context context, Action action)</a></li> +<li><a href="#Registering_Errors">Registering Errors</a></li></ul></li> +<li><a href="#Deploying_a_custom_Action_Executor">Deploying a custom Action Executor</a></li></ul> + +<div class="section"> +<h2><a name="Introduction"></a>Introduction</h2> +<p>Oozie can be extended to support additional action types by writing a custom <a href="WorkflowFunctionalSpec.html#ActionNodes">Action Node</a>. Action Nodes can be synchronous or asynchronous.</p> +<ul> + +<li>Synchronous Node - Sync nodes are executed inline by Oozie, which waits for completion of these nodes before proceeding. Hence, these nodes should almost never be used and are meant for lightweight tasks like FileSystem move, mkdir, delete.</li> +<li>Asynchronous Nodes - Oozie starts asynchronous nodes, and then monitors the action being executed for completion. This is done via a callback from the action or Oozie polling for the action status.</li> +</ul></div> +<div class="section"> +<h2><a name="Writing_a_custom_Action_Node"></a>Writing a custom Action Node</h2> +<p>Action Executors are configured in the oozie configuration file oozie-site.xml. These executors are loaded during Oozie startup. <a href="DG_CustomActionExecutor.html#Deploying_a_custom_Action_Executor">Deploying a Custom Action Executor</a>.</p> +<p>Action Executors MUST extend the <tt>ActionExecutor</tt> class and override the required methods.</p> +<p>Most methods take as argument the Execution Context and the actual Action object with various configuration properties resolved.</p> +<div class="section"> +<h3><a name="ActionExecutor.Context"></a>ActionExecutor.Context</h3> +<p>The Execution context gives Action Nodes access to configuration properties, methods to set the state of the action, methods to set variables which are to be made available later in the execution path.</p> +<p><b>The following methods from the ActionExecutor interface should be implemented.</b></p></div> +<div class="section"> +<h3><a name="Constructor"></a>Constructor</h3> +<p>A no argument constructor should be implemented, which calls super(ACTION_TYPE). ACTION_TYPE is the name of the action which will be used in the workflow xml, and is used by Oozie to instantiate the correct type of Executor.</p></div> +<div class="section"> +<h3><a name="initActionType"></a>initActionType()</h3> +<p>This method is called once during initialization of the Action Executor during Oozie startup. Any common initialization code for the Action Node should go here.</p> +<p>As an example, setting up of error handling for the Custom Action should be done here.</p> +<p>This method must call super.initActionType() as it’s first statement.</p></div> +<div class="section"> +<h3><a name="startActionExecutor.Context_context_Action_action"></a>start(ActionExecutor.Context context, Action action)</h3> +<p>The action start up happens here.</p> +<ul> + +<li>Async Actions - The action should be started and context.setStartData(externalId, trackerUri, consoleUrl) must be set. A check can be made for whether the action has completed, in which case context.setExecutionData(externalStatus, actionData) must be called.</li> +<li>Sync Actions - The action should be started and should complete execution. context.setExecutionData(externalStatus, actionData) must be called.</li> +</ul></div> +<div class="section"> +<h3><a name="checkActionExecutor.Context_context_Action_action"></a>check(ActionExecutor.Context context, Action action)</h3> +<p><tt>check(...)</tt> is used by Oozie to poll for the status of the action. This method should interact with the action started previously, and update the status. If the action has completed, context.setExecutionData(externalStatus, actionData) must be called. Otherwise, the status can be updated using context.setExternalStatus(externalStatus).</p> +<p>For sync actions, this method will not be called, and should throw an UnsupportedOperationException().</p></div> +<div class="section"> +<h3><a name="killActionExecutor.Context_context_Action_action"></a>kill(ActionExecutor.Context context, Action action)</h3> +<p><tt>kill(...)</tt> is called when there is an attempt to kill the running job or action. No workflow transition is made after this.</p> +<p>The implementation for a custom action should interact with and kill the running action, and take care of any cleanup which may be required. context.setEndData(status, signalValue) should be called with both values set to Action.Status.KILLED.</p></div> +<div class="section"> +<h3><a name="endActionExecutor.Context_context_Action_action"></a>end(ActionExecutor.Context context, Action action)</h3> +<p><tt>end(...)</tt> is used for any cleanup or processing which may need to be done after completion of the action. After any processing, context.setEndData(status, signalValue) should be called to complete execution of the action and trigger the next workflow transition. signalValue can be Action.Status.OK or Action.Status.ERROR.</p></div> +<div class="section"> +<h3><a name="Registering_Errors"></a>Registering Errors</h3> +<p>Oozie actions can generate different types of Errors.</p> +<ul> + +<li>TRANSIENT - will be retried</li> +<li>NON TRANSIENT - the job will be suspended and can be resumed later by human intervention, after fixing whatever problem caused this error.</li> +<li>ERROR - causes the error transition to be taken.</li> +<li>FAILED - the action and the job are set to FAILED state. No transitions are taken. registerError(exceptionClassName, errorType, errorMessage) can be used to register possible exceptions while executing the action, along with their type and error message. This will normally be done during initialization of the Action Executor.</li> +</ul></div></div> +<div class="section"> +<h2><a name="Deploying_a_custom_Action_Executor"></a>Deploying a custom Action Executor</h2> +<p>Action Nodes can be registered in the oozie configuration file oozie-site.xml, by changing the property ‘oozie.service.ActionService.executor.ext.classes’. For multiple Executors, the class name should be separated by commas.</p> + +<div> +<div> +<pre class="source"> <name>oozie.service.ActionService.executor.ext.classes</name> + <value> + org.apache.oozie.wf.action.decision.CustomActionExecutor, + Custom_Action_Executr_2.class + </value> + </property> +</pre></div></div> + +<p>Any configuration properties to be made available to this class should also be added to oozie-site.xml. The convention to be followed for naming these properties is ‘oozie.action.[ActionName].property.name’</p> +<p>The XML schema (XSD) for the new Actions should be added to oozie-site.xml, under the property ‘oozie.service.WorkflowSchemaService.ext.schemas’. A comma separated list for multiple Action schemas.</p> +<p>The XML schema (XSD) for the new action should be also added to Fluent Job API. Please refer to <a href="DG_FluentJobAPI.html#AE.C_Appendix_C_How_To_Extend">Fluent Job API :: How To Extend</a> for details.</p> +<p>The executor class should be placed along with the oozie webapp in the correct path. Once Oozie is restarted, the custom action node can be used in workflows.</p> +<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row-fluid"> + <p >Copyright © 2021 + <a href="https://www.apache.org";>Apache Software Foundation</a>. + All rights reserved. + + </p> + </div> + + + </div> + </footer> + </body> +</html> Added: oozie/site/trunk/content/resources/docs/5.2.1/DG_DistCpActionExtension.html URL: http://svn.apache.org/viewvc/oozie/site/trunk/content/resources/docs/5.2.1/DG_DistCpActionExtension.html?rev=1886952&view=auto ============================================================================== --- oozie/site/trunk/content/resources/docs/5.2.1/DG_DistCpActionExtension.html (added) +++ oozie/site/trunk/content/resources/docs/5.2.1/DG_DistCpActionExtension.html Fri Feb 26 14:14:19 2021 @@ -0,0 +1,334 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2021-02-26 + | Rendered using Apache Maven Fluido Skin 1.4 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20210226" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Oozie – </title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> + + + </head> + <body class="topBarDisabled"> + + + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="https://oozie.apache.org/"; id="bannerLeft"> + <img src="https://oozie.apache.org/images/oozie_200x.png"; alt="Oozie"/> + </a> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="http://www.apache.org/"; class="externalLink" title="Apache"> + Apache</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../../" title="Oozie"> + Oozie</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../" title="docs"> + docs</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="./" title="5.2.1"> + 5.2.1</a> + <span class="divider">/</span> + </li> + <li class="active "></li> + + + + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2021-02-26</li> + <li id="projectVersion" class="pull-right"> + Version: 5.2.1 + </li> + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span2"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + </ul> + + + + <hr /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span10" > + + <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p><hr /> +<h1>Oozie DistCp Action Extension</h1> +<ul> +<li><a href="#DistCp_Action">DistCp Action</a></li> +<li><a href="#Appendix_DistCp_XML-Schema">Appendix, DistCp XML-Schema</a> +<ul> +<li><a href="#AE.A_Appendix_A_DistCp_XML-Schema">AE.A Appendix A, DistCp XML-Schema</a> +<ul> +<li><a href="#DistCp_Action_Schema_Version_1.0">DistCp Action Schema Version 1.0</a></li> +<li><a href="#DistCp_Action_Schema_Version_0.2">DistCp Action Schema Version 0.2</a></li> +<li><a href="#DistCp_Action_Schema_Version_0.1">DistCp Action Schema Version 0.1</a></li></ul></li></ul></li></ul> + +<div class="section"> +<h2><a name="DistCp_Action"></a>DistCp Action</h2> +<p>The <tt>DistCp</tt> action uses Hadoop distributed copy to copy files from one cluster to another or within the same cluster.</p> +<p><b>IMPORTANT:</b> The DistCp action may not work properly with all configurations (secure, insecure) in all versions of Hadoop. For example, distcp between two secure clusters is tested and works well. Same is true with two insecure clusters. In cases where a secure and insecure clusters are involved, distcp will not work.</p> +<p>Both Hadoop clusters have to be configured with proxyuser for the Oozie process as explained <a href="DG_QuickStart.html#HadoopProxyUser">here</a> on the Quick Start page.</p> +<p><b>Syntax:</b></p> + +<div> +<div> +<pre class="source"><workflow-app name="[WF-DEF-NAME]" xmlns="uri:oozie:workflow:1.0"> + ... + <action name="distcp-example"> + <distcp xmlns="uri:oozie:distcp-action:1.0"> + <resource-manager>${resourceManager}</resource-manager> + <name-node>${nameNode1}</name-node> + <arg>${nameNode1}/path/to/input.txt</arg> + <arg>${nameNode2}/path/to/output.txt</arg> + </distcp> + <ok to="end"/> + <error to="fail"/> + </action> + ... +</workflow-app> +</pre></div></div> + +<p>The first <tt>arg</tt> indicates the input and the second <tt>arg</tt> indicates the output. In the above example, the input is on <tt>namenode1</tt> and the output is on <tt>namenode2</tt>.</p> +<p><b>IMPORTANT:</b> If using the DistCp action between 2 secure clusters, the following property must be added to the <tt>configuration</tt> of the action:</p> + +<div> +<div> +<pre class="source"><property> + <name>oozie.launcher.mapreduce.job.hdfs-servers</name> + <value>${nameNode1},${nameNode2}</value> +</property> +</pre></div></div> + +<p>The <tt>DistCp</tt> action is also commonly used to copy files within the same cluster. Cases where copying files within a directory to another directory or directories to target directory is supported. Example below will illustrate a copy within a cluster, notice the source and target <tt>nameNode</tt> is the same and use of <tt>*</tt> syntax is supported to represent only child files or directories within a source directory. For the sake of the example, <tt>jobTracker</tt> and <tt>resourceManager</tt> are synonymous.</p> +<p><b>Syntax:</b></p> + +<div> +<div> +<pre class="source"><workflow-app name="[WF-DEF-NAME]" xmlns="uri:oozie:workflow:1.0"> + ... + <action name="copy-example"> + <distcp xmlns="uri:oozie:distcp-action:1.0"> + <resource-manager>${resourceManager}</resource-manager> + <name-node>${nameNode}</name-node> + <arg>${nameNode}/path/to/source/*</arg> + <arg>${nameNode}/path/to/target/</arg> + </distcp> + <ok to="end"/> + <error to="fail"/> + </action> + ... +</workflow-app> +</pre></div></div> +</div> +<div class="section"> +<h2><a name="Appendix_DistCp_XML-Schema"></a>Appendix, DistCp XML-Schema</h2> +<div class="section"> +<h3><a name="AE.A_Appendix_A_DistCp_XML-Schema"></a>AE.A Appendix A, DistCp XML-Schema</h3> +<div class="section"> +<h4><a name="DistCp_Action_Schema_Version_1.0"></a>DistCp Action Schema Version 1.0</h4> + +<div> +<div> +<pre class="source"><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" + xmlns:distcp="uri:oozie:distcp-action:1.0" elementFormDefault="qualified" + targetNamespace="uri:oozie:distcp-action:1.0"> +. + <xs:include schemaLocation="oozie-common-1.0.xsd"/> +. + <xs:element name="distcp" type="distcp:ACTION"/> +. + <xs:complexType name="ACTION"> + <xs:sequence> + <xs:choice> + <xs:element name="job-tracker" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="resource-manager" type="xs:string" minOccurs="0" maxOccurs="1"/> + </xs:choice> + <xs:element name="name-node" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="prepare" type="distcp:PREPARE" minOccurs="0" maxOccurs="1"/> + <xs:element name="launcher" type="distcp:LAUNCHER" minOccurs="0" maxOccurs="1"/> + <xs:element name="configuration" type="distcp:CONFIGURATION" minOccurs="0" maxOccurs="1"/> + <xs:element name="java-opts" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="arg" type="xs:string" minOccurs="0" maxOccurs="unbounded"/> + </xs:sequence> + </xs:complexType> +. +</xs:schema> +</pre></div></div> +</div> +<div class="section"> +<h4><a name="DistCp_Action_Schema_Version_0.2"></a>DistCp Action Schema Version 0.2</h4> + +<div> +<div> +<pre class="source"><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" + xmlns:distcp="uri:oozie:distcp-action:0.2" elementFormDefault="qualified" + targetNamespace="uri:oozie:distcp-action:0.2"> +. + <xs:element name="distcp" type="distcp:ACTION"/> +. + <xs:complexType name="ACTION"> + <xs:sequence> + <xs:element name="job-tracker" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="name-node" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="prepare" type="distcp:PREPARE" minOccurs="0" maxOccurs="1"/> + <xs:element name="configuration" type="distcp:CONFIGURATION" minOccurs="0" maxOccurs="1"/> + <xs:element name="java-opts" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="arg" type="xs:string" minOccurs="0" maxOccurs="unbounded"/> + </xs:sequence> + </xs:complexType> +. + <xs:complexType name="CONFIGURATION"> + <xs:sequence> + <xs:element name="property" minOccurs="1" maxOccurs="unbounded"> + <xs:complexType> + <xs:sequence> + <xs:element name="name" minOccurs="1" maxOccurs="1" type="xs:string"/> + <xs:element name="value" minOccurs="1" maxOccurs="1" type="xs:string"/> + <xs:element name="description" minOccurs="0" maxOccurs="1" type="xs:string"/> + </xs:sequence> + </xs:complexType> + </xs:element> + </xs:sequence> + </xs:complexType> +. + <xs:complexType name="PREPARE"> + <xs:sequence> + <xs:element name="delete" type="distcp:DELETE" minOccurs="0" maxOccurs="unbounded"/> + <xs:element name="mkdir" type="distcp:MKDIR" minOccurs="0" maxOccurs="unbounded"/> + </xs:sequence> + </xs:complexType> +. + <xs:complexType name="DELETE"> + <xs:attribute name="path" type="xs:string" use="required"/> + </xs:complexType> +. + <xs:complexType name="MKDIR"> + <xs:attribute name="path" type="xs:string" use="required"/> + </xs:complexType> +. +</xs:schema> +</pre></div></div> +</div> +<div class="section"> +<h4><a name="DistCp_Action_Schema_Version_0.1"></a>DistCp Action Schema Version 0.1</h4> + +<div> +<div> +<pre class="source"><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" + xmlns:distcp="uri:oozie:distcp-action:0.1" elementFormDefault="qualified" + targetNamespace="uri:oozie:distcp-action:0.1"> +. + <xs:element name="distcp" type="distcp:ACTION"/> +. + <xs:complexType name="ACTION"> + <xs:sequence> + <xs:element name="job-tracker" type="xs:string" minOccurs="1" maxOccurs="1"/> + <xs:element name="name-node" type="xs:string" minOccurs="1" maxOccurs="1"/> + <xs:element name="prepare" type="distcp:PREPARE" minOccurs="0" maxOccurs="1"/> + <xs:element name="configuration" type="distcp:CONFIGURATION" minOccurs="0" maxOccurs="1"/> + <xs:element name="java-opts" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="arg" type="xs:string" minOccurs="0" maxOccurs="unbounded"/> + </xs:sequence> + </xs:complexType> +. + <xs:complexType name="CONFIGURATION"> + <xs:sequence> + <xs:element name="property" minOccurs="1" maxOccurs="unbounded"> + <xs:complexType> + <xs:sequence> + <xs:element name="name" minOccurs="1" maxOccurs="1" type="xs:string"/> + <xs:element name="value" minOccurs="1" maxOccurs="1" type="xs:string"/> + <xs:element name="description" minOccurs="0" maxOccurs="1" type="xs:string"/> + </xs:sequence> + </xs:complexType> + </xs:element> + </xs:sequence> + </xs:complexType> +. + <xs:complexType name="PREPARE"> + <xs:sequence> + <xs:element name="delete" type="distcp:DELETE" minOccurs="0" maxOccurs="unbounded"/> + <xs:element name="mkdir" type="distcp:MKDIR" minOccurs="0" maxOccurs="unbounded"/> + </xs:sequence> + </xs:complexType> +. + <xs:complexType name="DELETE"> + <xs:attribute name="path" type="xs:string" use="required"/> + </xs:complexType> +. + <xs:complexType name="MKDIR"> + <xs:attribute name="path" type="xs:string" use="required"/> + </xs:complexType> +. +</xs:schema> +</pre></div></div> + +<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div></div></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row-fluid"> + <p >Copyright © 2021 + <a href="https://www.apache.org";>Apache Software Foundation</a>. + All rights reserved. + + </p> + </div> + + + </div> + </footer> + </body> +</html> Added: oozie/site/trunk/content/resources/docs/5.2.1/DG_EmailActionExtension.html URL: http://svn.apache.org/viewvc/oozie/site/trunk/content/resources/docs/5.2.1/DG_EmailActionExtension.html?rev=1886952&view=auto ============================================================================== --- oozie/site/trunk/content/resources/docs/5.2.1/DG_EmailActionExtension.html (added) +++ oozie/site/trunk/content/resources/docs/5.2.1/DG_EmailActionExtension.html Fri Feb 26 14:14:19 2021 @@ -0,0 +1,234 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2021-02-26 + | Rendered using Apache Maven Fluido Skin 1.4 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20210226" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Oozie – </title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> + + + </head> + <body class="topBarDisabled"> + + + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="https://oozie.apache.org/"; id="bannerLeft"> + <img src="https://oozie.apache.org/images/oozie_200x.png"; alt="Oozie"/> + </a> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="http://www.apache.org/"; class="externalLink" title="Apache"> + Apache</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../../" title="Oozie"> + Oozie</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../" title="docs"> + docs</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="./" title="5.2.1"> + 5.2.1</a> + <span class="divider">/</span> + </li> + <li class="active "></li> + + + + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2021-02-26</li> + <li id="projectVersion" class="pull-right"> + Version: 5.2.1 + </li> + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span2"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + </ul> + + + + <hr /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span10" > + + <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p><hr /> +<h1>Oozie Email Action Extension</h1> +<ul> +<li><a href="#a3.2.4_Email_action">3.2.4 Email action</a></li> +<li><a href="#AE.A_Appendix_A_Email_XML-Schema">AE.A Appendix A, Email XML-Schema</a></li></ul> + +<p><a name="EmailAction"></a></p> +<div class="section"> +<h2><a name="a3.2.4_Email_action"></a>3.2.4 Email action</h2> +<p>The <tt>email</tt> action allows sending emails in Oozie from a workflow application. An email action must provide <tt>to</tt> addresses, <tt>cc</tt> addresses (optional), <tt>bcc</tt> addresses (optional), a <tt>subject</tt> and a <tt>body</tt>. Multiple recipients of an email can be provided as comma separated addresses.</p> +<p>The email action is executed synchronously, and the workflow job will wait until the specified emails are sent before continuing to the next action.</p> +<p>All values specified in the <tt>email</tt> action can be parameterized (templatized) using EL expressions.</p> +<p><b>Syntax:</b></p> + +<div> +<div> +<pre class="source"><workflow-app name="[WF-DEF-NAME]" xmlns="uri:oozie:workflow:0.1"> + ... + <action name="[NODE-NAME]"> + <email xmlns="uri:oozie:email-action:0.2"> + <to>[COMMA-SEPARATED-TO-ADDRESSES]</to> + <cc>[COMMA-SEPARATED-CC-ADDRESSES]</cc> <!-- cc is optional --> + <bcc>[COMMA-SEPARATED-BCC-ADDRESSES]</bcc> <!-- bcc is optional --> + <subject>[SUBJECT]</subject> + <body>[BODY]</body> + <content_type>[CONTENT-TYPE]</content_type> <!-- content_type is optional --> + <attachment>[COMMA-SEPARATED-HDFS-FILE-PATHS]</attachment> <!-- attachment is optional --> + </email> + <ok to="[NODE-NAME]"/> + <error to="[NODE-NAME]"/> + </action> + ... +</workflow-app> +</pre></div></div> + +<p>The <tt>to</tt> and <tt>cc</tt> and <tt>bcc</tt> commands are used to specify recipients who should get the mail. Multiple email recipients can be provided using comma-separated values. Providing a <tt>to</tt> command is necessary, while the <tt>cc</tt> or <tt>bcc</tt> may optionally be used along.</p> +<p>The <tt>subject</tt> and <tt>body</tt> commands are used to specify subject and body of the mail. From uri:oozie:email-action:0.2 one can also specify mail content type as <content_type>text/html</content_type>. “text/plain” is default.</p> +<p>The <tt>attachment</tt> is used to attach a file(s) on HDFS to the mail. Multiple attachment can be provided using comma-separated values. Non fully qualified path is considered as a file on default HDFS. A local file cannot be attached.</p> +<p><b>Configuration</b></p> +<p>The <tt>email</tt> action requires some SMTP server configuration to be present (in oozie-site.xml). The following are the values it looks for:</p> +<ul> + +<li><tt>oozie.email.smtp.host</tt> - The host where the email action may find the SMTP server (localhost by default).</li> +<li><tt>oozie.email.smtp.port</tt> - The port to connect to for the SMTP server (25 by default).</li> +<li><tt>oozie.email.from.address</tt> - The from address to be used for mailing all emails (oozie@localhost by default).</li> +<li><tt>oozie.email.smtp.auth</tt> - Boolean property that toggles if authentication is to be done or not. (false by default).</li> +<li><tt>oozie.email.smtp.starttls.enable</tt> - Boolean property that toggles if use TLS communication or not. (false by default).</li> +<li><tt>oozie.email.smtp.username</tt> - If authentication is enabled, the username to login as (empty by default).</li> +<li><tt>oozie.email.smtp.password</tt> - If authentication is enabled, the username’s password (empty by default).</li> +<li><tt>oozie.email.attachment.enabled</tt> - Boolean property that toggles if configured attachments are to be placed into the emails. (false by default).</li> +<li><tt>oozie.email.smtp.socket.timeout.ms</tt> - The timeout to apply over all SMTP server socket operations (10000ms by default).</li> +</ul> +<p><b>Example:</b></p> + +<div> +<div> +<pre class="source"><workflow-app name="sample-wf" xmlns="uri:oozie:workflow:0.1"> + ... + <action name="an-email"> + <email xmlns="uri:oozie:email-action:0.1"> + <to>b...@initech.com,the.other....@initech.com</to> + <cc>w...@initech.com</cc> + <bcc>yet.another....@initech.com</bcc> + <subject>Email notifications for ${wf:id()}</subject> + <body>The wf ${wf:id()} successfully completed.</body> + </email> + <ok to="myotherjob"/> + <error to="errorcleanup"/> + </action> + ... +</workflow-app> +</pre></div></div> + +<p>In the above example, an email is sent to ‘bob’, ‘the.other.bob’, ‘will’ (cc), yet.another.bob (bcc) with the subject and body both containing the workflow ID after substitution.</p></div> +<div class="section"> +<h2><a name="AE.A_Appendix_A_Email_XML-Schema"></a>AE.A Appendix A, Email XML-Schema</h2> + +<div> +<div> +<pre class="source"><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" + xmlns:email="uri:oozie:email-action:0.2" elementFormDefault="qualified" + targetNamespace="uri:oozie:email-action:0.2"> +. + <xs:element name="email" type="email:ACTION"/> +. + <xs:complexType name="ACTION"> + <xs:sequence> + <xs:element name="to" type="xs:string" minOccurs="1" maxOccurs="1"/> + <xs:element name="cc" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="bcc" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="subject" type="xs:string" minOccurs="1" maxOccurs="1"/> + <xs:element name="body" type="xs:string" minOccurs="1" maxOccurs="1"/> + <xs:element name="content_type" type="xs:string" minOccurs="0" maxOccurs="1"/> + <xs:element name="attachment" type="xs:string" minOccurs="0" maxOccurs="1"/> + </xs:sequence> + </xs:complexType> +</xs:schema> +</pre></div></div> + +<p><b>GMail example to oozie-site.xml</b></p> + +<div> +<div> +<pre class="source">oozie.email.smtp.host=smtp.gmail.com +oozie.email.smtp.port=587 +oozie.email.from.address=<some email address> +oozie.email.smtp.auth=true +oozie.email.smtp.starttls.enable=true +oozie.email.smtp.username=<Gmail Id> +oozie.email.smtp.password=<Gmail Pass> +</pre></div></div> + +<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row-fluid"> + <p >Copyright © 2021 + <a href="https://www.apache.org";>Apache Software Foundation</a>. + All rights reserved. + + </p> + </div> + + + </div> + </footer> + </body> +</html> Added: oozie/site/trunk/content/resources/docs/5.2.1/DG_Examples.html URL: http://svn.apache.org/viewvc/oozie/site/trunk/content/resources/docs/5.2.1/DG_Examples.html?rev=1886952&view=auto ============================================================================== --- oozie/site/trunk/content/resources/docs/5.2.1/DG_Examples.html (added) +++ oozie/site/trunk/content/resources/docs/5.2.1/DG_Examples.html Fri Feb 26 14:14:19 2021 @@ -0,0 +1,296 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2021-02-26 + | Rendered using Apache Maven Fluido Skin 1.4 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20210226" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Oozie – </title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> + + + </head> + <body class="topBarDisabled"> + + + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="https://oozie.apache.org/"; id="bannerLeft"> + <img src="https://oozie.apache.org/images/oozie_200x.png"; alt="Oozie"/> + </a> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="http://www.apache.org/"; class="externalLink" title="Apache"> + Apache</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../../" title="Oozie"> + Oozie</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="../" title="docs"> + docs</a> + <span class="divider">/</span> + </li> + <li class=""> + <a href="./" title="5.2.1"> + 5.2.1</a> + <span class="divider">/</span> + </li> + <li class="active "></li> + + + + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2021-02-26</li> + <li id="projectVersion" class="pull-right"> + Version: 5.2.1 + </li> + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span2"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + </ul> + + + + <hr /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span10" > + + <p><a href="index.html">::Go back to Oozie Documentation Index::</a></p> +<h1>Oozie Examples</h1> +<ul> +<li><a href="#Command_Line_Examples">Command Line Examples</a> +<ul> +<li><a href="#Setting_Up_the_Examples">Setting Up the Examples</a></li> +<li><a href="#Running_the_Examples">Running the Examples</a></li></ul></li> +<li><a href="#Java_API_Example">Java API Example</a></li> +<li><a href="#Local_Oozie_Example">Local Oozie Example</a></li> +<li><a href="#Fluent_Job_API_Examples">Fluent Job API Examples</a></li></ul> + +<div class="section"> +<h2><a name="Command_Line_Examples"></a>Command Line Examples</h2> +<div class="section"> +<h3><a name="Setting_Up_the_Examples"></a>Setting Up the Examples</h3> +<p>Oozie examples are bundled within the Oozie distribution in the <tt>oozie-examples.tar.gz</tt> file.</p> +<p>Expanding this file will create an <tt>examples/</tt> directory in the local file system.</p> +<p>The <tt>examples/</tt> directory must be copied to the user HOME directory in HDFS:</p> + +<div> +<div> +<pre class="source">$ hadoop fs -put examples examples +</pre></div></div> + +<p><b>NOTE:</b> If an examples directory already exists in HDFS, it must be deleted before copying it again. Otherwise files may not be copied.</p></div> +<div class="section"> +<h3><a name="Running_the_Examples"></a>Running the Examples</h3> +<p>For the Streaming and Pig example, the <a href="DG_QuickStart.html#OozieShareLib">Oozie Share Library</a> must be installed in HDFS.</p> +<p>Add Oozie <tt>bin/</tt> to the environment PATH.</p> +<p>The examples assume the ResourceManager is <tt>localhost:8032</tt> and the NameNode is <tt>hdfs://localhost:8020</tt>. If the actual values are different, the job properties files in the examples directory must be edited to the correct values.</p> +<p>The example applications are under the examples/app directory, one directory per example. The directory contains the application XML file (workflow, or workflow and coordinator), the <tt>job.properties</tt> file to submit the job and any JAR files the example may need.</p> +<p>The inputs for all examples are in the <tt>examples/input-data/</tt> directory.</p> +<p>The examples create output under the <tt>examples/output-data/${EXAMPLE_NAME}</tt> directory.</p> +<p><b>Note</b>: The <tt>job.properties</tt> file needs to be a local file during submissions, and not a HDFS path.</p> +<p><b>How to run an example application:</b></p> + +<div> +<div> +<pre class="source">$ oozie job -oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -run +. +job: 14-20090525161321-oozie-tucu +</pre></div></div> + +<p>Check the workflow job status:</p> + +<div> +<div> +<pre class="source">$ oozie job -oozie http://localhost:11000/oozie -info 14-20090525161321-oozie-tucu +. +.---------------------------------------------------------------------------------------------------------------------------------------------------------------- +Workflow Name : map-reduce-wf +App Path : hdfs://localhost:8020/user/tucu/examples/apps/map-reduce +Status : SUCCEEDED +Run : 0 +User : tucu +Group : users +Created : 2009-05-26 05:01 +0000 +Started : 2009-05-26 05:01 +0000 +Ended : 2009-05-26 05:01 +0000 +Actions +.---------------------------------------------------------------------------------------------------------------------------------------------------------------- +Action Name Type Status Transition External Id External Status Error Code Start Time End Time +.---------------------------------------------------------------------------------------------------------------------------------------------------------------- +mr-node map-reduce OK end job_200904281535_0254 SUCCEEDED - 2009-05-26 05:01 +0000 2009-05-26 05:01 +0000 +.---------------------------------------------------------------------------------------------------------------------------------------------------------------- +</pre></div></div> + +<p>To check the workflow job status via the Oozie web console, with a browser go to <tt>http://localhost:11000/oozie</tt>.</p> +<p>To avoid having to provide the <tt>-oozie</tt> option with the Oozie URL with every <tt>oozie</tt> command, set <tt>OOZIE_URL</tt> env variable to the Oozie URL in the shell environment. For example:</p> + +<div> +<div> +<pre class="source">$ export OOZIE_URL="http://localhost:11000/oozie" +$ +$ oozie job -info 14-20090525161321-oozie-tucu +</pre></div></div> +</div></div> +<div class="section"> +<h2><a name="Java_API_Example"></a>Java API Example</h2> +<p>Oozie provides a <a href="./apidocs/org/org/apache/oozie/client/package-summary.html">Java Client API</a> that simplifies integrating Oozie with Java applications. This Java Client API is a convenience API to interact with Oozie Web-Services API.</p> +<p>The following code snippet shows how to submit an Oozie job using the Java Client API.</p> + +<div> +<div> +<pre class="source">import org.apache.oozie.client.OozieClient; +import org.apache.oozie.client.WorkflowJob; +. +import java.util.Properties; +. + ... +. + // get a OozieClient for local Oozie + OozieClient wc = new OozieClient("http://bar:11000/oozie"); +. + // create a workflow job configuration and set the workflow application path + Properties conf = wc.createConfiguration(); + conf.setProperty(OozieClient.APP_PATH, "hdfs://foo:8020/usr/tucu/my-wf-app"); +. + // setting workflow parameters + conf.setProperty("resourceManager", "foo:8032"); + conf.setProperty("inputDir", "/usr/tucu/inputdir"); + conf.setProperty("outputDir", "/usr/tucu/outputdir"); + ... +. + // submit and start the workflow job + String jobId = wc.run(conf); + System.out.println("Workflow job submitted"); +. + // wait until the workflow job finishes printing the status every 10 seconds + while (wc.getJobInfo(jobId).getStatus() == Workflow.Status.RUNNING) { + System.out.println("Workflow job running ..."); + Thread.sleep(10 * 1000); + } +. + // print the final status of the workflow job + System.out.println("Workflow job completed ..."); + System.out.println(wf.getJobInfo(jobId)); + ... +</pre></div></div> +</div> +<div class="section"> +<h2><a name="Local_Oozie_Example"></a>Local Oozie Example</h2> +<p>Oozie provides an embedded Oozie implementation, <a href="./apidocs/org/apache/oozie/local/LocalOozie.html">LocalOozie</a> , which is useful for development, debugging and testing of workflow applications within the convenience of an IDE.</p> +<p>The code snippet below shows the usage of the <tt>LocalOozie</tt> class. All the interaction with Oozie is done using Oozie <tt>OozieClient</tt> Java API, as shown in the previous section.</p> +<p>The examples bundled with Oozie include the complete and running class, <tt>LocalOozieExample</tt> from where this snippet was taken.</p> + +<div> +<div> +<pre class="source">import org.apache.oozie.local.LocalOozie; +import org.apache.oozie.client.OozieClient; +import org.apache.oozie.client.WorkflowJob; +. +import java.util.Properties; +. + ... + // start local Oozie + LocalOozie.start(); +. + // get a OozieClient for local Oozie + OozieClient wc = LocalOozie.getClient(); +. + // create a workflow job configuration and set the workflow application path + Properties conf = wc.createConfiguration(); + conf.setProperty(OozieClient.APP_PATH, "hdfs://foo:8020/usr/tucu/my-wf-app"); +. + // setting workflow parameters + conf.setProperty("resourceManager", "foo:8032"); + conf.setProperty("inputDir", "/usr/tucu/inputdir"); + conf.setProperty("outputDir", "/usr/tucu/outputdir"); + ... +. + // submit and start the workflow job + String jobId = wc.run(conf); + System.out.println("Workflow job submitted"); +. + // wait until the workflow job finishes printing the status every 10 seconds + while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) { + System.out.println("Workflow job running ..."); + Thread.sleep(10 * 1000); + } +. + // print the final status of the workflow job + System.out.println("Workflow job completed ..."); + System.out.println(wc.getJobInfo(jobId)); +. + // stop local Oozie + LocalOozie.stop(); + ... +</pre></div></div> + +<p>Also asynchronous actions like FS action can be used / tested using <tt>LocalOozie</tt> / <tt>OozieClient</tt> API. Please see the module <tt>oozie-mini</tt> for details like <tt>fs-decision.xml</tt> workflow example.</p></div> +<div class="section"> +<h2><a name="Fluent_Job_API_Examples"></a>Fluent Job API Examples</h2> +<p>There are some elaborate examples how to use the <a href="DG_FluentJobAPI.html">Fluent Job API</a>, under <tt>examples/fluentjob/</tt>. There are two simple examples covered under <a href="DG_FluentJobAPI.html#A_Simple_Example">Fluent Job API :: A Simple Example</a> and <a href="DG_FluentJobAPI.html#A_More_Verbose_Example">Fluent Job API :: A More Verbose Example</a>.</p> +<p><a href="index.html">::Go back to Oozie Documentation Index::</a></p></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row-fluid"> + <p >Copyright © 2021 + <a href="https://www.apache.org";>Apache Software Foundation</a>. + All rights reserved. + + </p> + </div> + + + </div> + </footer> + </body> +</html>
![https://oozie.apache.org/images/oozie_200x.png"](https://oozie.apache.org/images/oozie_200x.png"){kind=link}