Repository: oozie Updated Branches: refs/heads/master 81c323e2e -> 2cdec0e4a
OOZIE-3342 [examples] Missing archive tag from Fluent Job examples (asalamon74 via andras.piros) Project: http://git-wip-us.apache.org/repos/asf/oozie/repo Commit: http://git-wip-us.apache.org/repos/asf/oozie/commit/2cdec0e4 Tree: http://git-wip-us.apache.org/repos/asf/oozie/tree/2cdec0e4 Diff: http://git-wip-us.apache.org/repos/asf/oozie/diff/2cdec0e4 Branch: refs/heads/master Commit: 2cdec0e4ab3d51f0e1c19eacbc2c4dacda04f8ca Parents: 81c323e Author: Andras Piros <andras.pi...@cloudera.com> Authored: Fri Sep 21 10:52:03 2018 +0200 Committer: Andras Piros <andras.pi...@cloudera.com> Committed: Fri Sep 21 10:52:03 2018 +0200 ---------------------------------------------------------------------- docs/src/site/markdown/DG_FluentJobAPI.md | 107 +++++++++++++++++++ .../oozie/example/fluentjob/JavaMain.java | 2 + .../apache/oozie/example/fluentjob/Spark.java | 1 + release-log.txt | 1 + 4 files changed, 111 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/oozie/blob/2cdec0e4/docs/src/site/markdown/DG_FluentJobAPI.md ---------------------------------------------------------------------- diff --git a/docs/src/site/markdown/DG_FluentJobAPI.md b/docs/src/site/markdown/DG_FluentJobAPI.md index bd36517..7319970 100644 --- a/docs/src/site/markdown/DG_FluentJobAPI.md +++ b/docs/src/site/markdown/DG_FluentJobAPI.md @@ -311,6 +311,113 @@ public class MySecondWorkflowFactory implements WorkflowFactory { </workflow:workflow-app> ``` +### Running the JavaMain example + +Oozie contains several simple Fluent Job API examples. The JavaMain Fluent JOB API example generates a workflow +similar to the basic java-main example. + +The source code for all the Fluent Job API examples can be found in the `oozie-examples.tar.gz` file. + +#### Compilation + +To compile the examples we also need the `oozie-fluent-job-api` jar file. +The name of the file also contains the Oozie version number, in this example we assume that the name of the file is +`oozie-fluent-job-api-5.1.0.jar`. + +Assuming that the `src` directory contains the source files we can use the following command to compile the JavaMain example: + +``` +javac -classpath oozie-fluent-job-api-5.1.0.jar src/org/apache/oozie/example/fluentjob/JavaMain.java +``` + +#### Jar creation + +The next command creates the jar file: + +``` +jar cfe fluentjob-javamain-example.jar org.apache.oozie.example.fluentjob.JavaMain -C src \ +org/apache/oozie/example/fluentjob/JavaMain.class +``` + +This jar contains only the `JavaMain.class` file. The content of the `MAINFEST.MF` file is the following: + +``` +Manifest-Version: 1.0 +Created-By: 1.8.0_171 (Oracle Corporation) +Main-Class: org.apache.oozie.example.fluentjob.JavaMain +``` + +#### Validating the jar + +It is possible to validate the jar file: + +``` +oozie job -oozie http://localhost:11000/oozie -validatejar fluentjob-javamain-example.jar +``` + +The command should print out: `Valid workflow-app` + +If we also use the `-verbose` option the command prints out the generated XML as well: + +``` +<?xml version="1.0" encoding="UTF-8" standalone="yes"?> +<workflow:workflow-app xmlns:email="uri:oozie:email-action:0.2" xmlns:workflow="uri:oozie:workflow:1.0" name="java-main-example"> + <workflow:start to="java-main"/> + <workflow:kill name="kill"> + <workflow:message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</workflow:message> + </workflow:kill> + <workflow:action name="email-on-error"> + <email:email> + <email:to>someb...@apache.org</email:to> + <email:subject>Workflow error</email:subject> + <email:body>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</email:body> + </email:email> + <workflow:ok to="kill"/> + <workflow:error to="kill"/> + </workflow:action> + <workflow:action name="java-main"> + <workflow:java> + <workflow:resource-manager>${resourceManager}</workflow:resource-manager> + <workflow:name-node>${nameNode}</workflow:name-node> + <workflow:configuration> + <workflow:property> + <workflow:name>mapred.job.queue.name</workflow:name> + <workflow:value>${queueName}</workflow:value> + </workflow:property> + </workflow:configuration> + <workflow:main-class>org.apache.oozie.example.DemoJavaMain</workflow:main-class> + <workflow:arg>Hello</workflow:arg> + <workflow:arg>Oozie!</workflow:arg> + <workflow:archive>${nameNode}/user/${wf:user()}/${examplesRoot}/apps/java-main/lib/oozie-examples-${projectVersion}.jar</workflow:archive> + </workflow:java> + <workflow:ok to="end"/> + <workflow:error to="email-on-error"/> + </workflow:action> + <workflow:end name="end"/> +</workflow:workflow-app> +``` + +### Running the jar + +To run the jar it is also necessary to provide a properties file using the `-config` option: + +``` +oozie job -oozie http://localhost:11000/oozie -runjar fluentjob-javamain-example.jar -config fluentjob-javamain.properties +``` + +The contents of the `fluentjob-javamain.properties` file is similar to the `job.properties` file of the basic java-main example, +but we also need one extra property called `projectVersion`. The following shows a sample properties file: + +``` +resourceManager=localhost:8032 +nameNode=hdfs://localhost:9000 +queueName=default +examplesRoot=examples +projectVersion=5.1.0 +``` + +It is also possible to use the `-verbose` option here if we want to print out the generated XML. + ### Runtime Limitations Even if Fluent Job API tries to abstract away the task of assembly job descriptor XML files, there are some runtime http://git-wip-us.apache.org/repos/asf/oozie/blob/2cdec0e4/examples/src/main/java/org/apache/oozie/example/fluentjob/JavaMain.java ---------------------------------------------------------------------- diff --git a/examples/src/main/java/org/apache/oozie/example/fluentjob/JavaMain.java b/examples/src/main/java/org/apache/oozie/example/fluentjob/JavaMain.java index a8948fd..7caf86e 100644 --- a/examples/src/main/java/org/apache/oozie/example/fluentjob/JavaMain.java +++ b/examples/src/main/java/org/apache/oozie/example/fluentjob/JavaMain.java @@ -38,6 +38,8 @@ public class JavaMain implements WorkflowFactory { .withNameNode("${nameNode}") .withConfigProperty("mapred.job.queue.name", "${queueName}") .withMainClass("org.apache.oozie.example.DemoJavaMain") + .withArchive( + "${nameNode}/user/${wf:user()}/${examplesRoot}/apps/java-main/lib/oozie-examples-${projectVersion}.jar") .withArg("Hello") .withArg("Oozie!") .withErrorHandler(ErrorHandler.buildAsErrorHandler(EmailActionBuilder.create() http://git-wip-us.apache.org/repos/asf/oozie/blob/2cdec0e4/examples/src/main/java/org/apache/oozie/example/fluentjob/Spark.java ---------------------------------------------------------------------- diff --git a/examples/src/main/java/org/apache/oozie/example/fluentjob/Spark.java b/examples/src/main/java/org/apache/oozie/example/fluentjob/Spark.java index 9527336..2bb0eb2 100644 --- a/examples/src/main/java/org/apache/oozie/example/fluentjob/Spark.java +++ b/examples/src/main/java/org/apache/oozie/example/fluentjob/Spark.java @@ -45,6 +45,7 @@ public class Spark implements WorkflowFactory { .withMode("${mode}") .withActionName("Spark File Copy Example") .withActionClass("org.apache.oozie.example.SparkFileCopy") + .withArchive("${nameNode}/user/${wf:user()}/${examplesRoot}/apps/spark/lib/oozie-examples.jar") .withJar("${nameNode}/user/${wf:user()}/${examplesRoot}/apps/spark/lib/oozie-examples.jar") .withErrorHandler(ErrorHandler.buildAsErrorHandler(EmailActionBuilder.create() .withName("email-on-error") http://git-wip-us.apache.org/repos/asf/oozie/blob/2cdec0e4/release-log.txt ---------------------------------------------------------------------- diff --git a/release-log.txt b/release-log.txt index 52b4f7c..7025185 100644 --- a/release-log.txt +++ b/release-log.txt @@ -1,5 +1,6 @@ -- Oozie 5.1.0 release (trunk - unreleased) +OOZIE-3342 [examples] Missing archive tag from Fluent Job examples (asalamon74 via andras.piros) OOZIE-2734 amend [docs] Switch from TWiki to Markdown (asalamon74 via andras.piros, pbacsko, gezapeti) OOZIE-2734 amend [docs] Switch from TWiki to Markdown (asalamon74 via andras.piros, pbacsko, gezapeti) OOZIE-2734 [docs] Switch from TWiki to Markdown (asalamon74 via andras.piros, pbacsko, gezapeti)
