Hi, thanks for the reply. I did hit many issues with MiniOoze from CDH 5.5.1 (Oozie 4.1 + some Cloudera patches on top of it) 1. It doesn't work without oozie git repo. I've fixed it. 2. I wasn't able to run LocalOoize. I'm using DagEngine directly.
dryrun works fine. And what if I want to execute my workflow with mocking actual actions? 2016-12-02 21:46 GMT+01:00 Robert Kanter <rkan...@cloudera.com>: > Hi Serega, > > It turns out that LocalOozieClient is missing some methods that it should > be implementing to call DagEngine (see OOZIE-2751 > <https://issues.apache.org/jira/browse/OOZIE-2751>). One of those is the > dryrun method, which is why that's not working for you. For now, calling > DagEngine directly should work fine (that's essentially what > LocalOozieClient > <https://github.com/apache/oozie/blob/master/core/src/ > main/java/org/apache/oozie/LocalOozieClient.java> > does). > > In your last line, dryRunSubmit doesn't return a jobId because it doesn't > actually start a job. It does a dryrun > <http://oozie.apache.org/docs/4.3.0/DG_CommandLineTool.html# > Dryrun_of_Workflow_Job>, > which essentially just checks that everything resolves correctly in the > workflow.xml without actually running any of the actions. If successful, > it returns the String "OK". If there's a problem, it throws an exception > that should contain the details of the problem. > > > - Robert > > On Fri, Dec 2, 2016 at 1:22 AM, Serega Sheypak <serega.shey...@gmail.com> > wrote: > > > Hi, did anyone make it work property in his project? > > I need to do dry run for my workflows. > > The usecase is: > > User writes workflow and wants to: > > 1. Check if it valid > > 2. do dryrun, see how it flows without executing steps. > > > > Let say I have wflow with three steps: > > > > 1. disctp data from $A to $B > > 2. run spark action with $B as input > > 3. disctp $B to $C > > > > I want to do dryrun and check how my variables were interpolated it > wflow. > > The killer feature is: I want to imitate spark action failure and check > how > > my kill node looks like. > > > > I gave up to make MiniOozie work. But I was able to start DagEngine. > > > > val fs: FileSystem = FileSystem.getLocal(new Configuration()) > > val appPath: Path = new Path("build", "app") > > fs.mkdirs(appPath) > > fs.mkdirs(new Path(appPath, "lib")) > > > > val writer = new OutputStreamWriter(fs.create(new Path(appPath, > > "workflow.xml"))) > > writer.write(pipelineXml) > > writer.close() > > > > val wc = getClient > > > > val workflowConfiguration = wc.createConfiguration > > workflowConfiguration.setProperty(OozieClient.APP_PATH, new > > Path(appPath, "workflow.xml").toString) > > workflowConfiguration.setProperty(OozieClient.USER_NAME, getTestUser) > > usrDefinedProperties.foreach { case (k, v) => > > workflowConfiguration.put(k.toString, v.toString) } > > > > val dagEngine:DagEngine = > > Services.get.get(classOf[DagEngineService]).getDagEngine(getTestUser) > > val conf = new Configuration() > > workflowConfiguration.asScala.toMap.foreach{case(k,v ) => conf.set(k, > > v)} > > val jobId = dagEngine.dryRunSubmit(conf) > > > > But I would like to check how workflow flows when you pass in different > > parameters without actualy executing steps. > > >
