----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38474/ -----------------------------------------------------------
(Updated Jan. 8, 2016, 2:32 a.m.) Review request for oozie. Bugs: OOZIE-1976 https://issues.apache.org/jira/browse/OOZIE-1976 Repository: oozie-git Description ------- There are three components in this patch 1. User interface A new tag is added to coordinator.xml ex. <input-check> <or name="test"> <and> <data-in dataset="A"/>" <data-in dataset="B"/> </and> <and> <data-in dataset="C"/> <data-in dataset="D"/> </and> </or>; <input-check> input-check will have nested and/or/combine operation. It can have min and wait at operator or at date-in. If input-check tag is missing then it consider to be old approach where all data dependency are needed. 2. Processing input-check is converted into logical expression (a&&B)||(c&&d) We use jexl to parse the logical expression. There are three phase in parsing. phase 1 : only resolved dataset are parsed ( only current). phase 2 : once all current are resolved, then future/latest are parsed. phase 3 : Doesn't do any filecheck, just return what is being parsed by phase1 and phase2. Is used for EL functions 3. Storage. if inputcheck is enable, push_missing_dependencies and missing_dependencies are serialized and stored in DB. If then not then it's old approach, where they are stored in plan text. This is backward compatible. Diffs (updated) ----- client/src/main/resources/oozie-coordinator-0.5.xsd e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/pom.xml b063dab79415447a86c1a33f5c3f5304e0dffca0 core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 91bff4dca2ef2dece68ca7260724ba3e43b1a08e core/src/main/java/org/apache/oozie/ErrorCode.java 6c1e3997c9a1cb0bb0de39d687a083af0a7b5f04 core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 742d00dd47aab55392abc8fbc207f8100728b832 core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdatePushMissingDependency.java 4e1c5b3392358cb6b1e98e16e469310338f27fed core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 58ef483272264e0d391d4a1e6533dc1cab9940da core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java 39e6ac15ce3a3ea7f2ed9178688537f7b1d7842d core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java b05344d89e8df0e11fe69c1aa725d19a18eb0a2b core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java d4d1c08141088e64e0551dccbd23a6185cfdbb7b core/src/main/java/org/apache/oozie/coord/CoordELConstants.java f010a817fc900821c0e429fc16e1d3902a98d8bb core/src/main/java/org/apache/oozie/coord/CoordELEvaluator.java 8b2f4560ae66bbcd707a446382e647663ea67be1 core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 5d238663aa94f0dd55a9190b60bfe621439c7b53 core/src/main/java/org/apache/oozie/coord/CoordUtils.java 94c69740618110ea180b188ab0c5a02db76f8b4d core/src/main/java/org/apache/oozie/coord/SyncCoordAction.java 44258eb5be40bf6769e32c8780117e8533d80d7e core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesChecker.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependency.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheck.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseOne.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseThree.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseTwo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseValidate.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyFactory.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputInstance.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordPullInputDependency.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordPushInputDependency.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/CoordUnResolvedInputDependency.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/InputLogicParser.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/OozieJexlEngine.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/dependency/OozieJexlInterpreter.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseThree.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseTwo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseValidate.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorResult.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/InputLogicParser.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/OozieJexlEngine.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/coord/input/logic/OozieJexlInterpreter.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/main/java/org/apache/oozie/dependency/ActionDependency.java c280d1dc2387c9b3df96d4e83ad5563e10b1e1ef core/src/main/java/org/apache/oozie/dependency/DependencyChecker.java a5507575a3403103133f85a78c873c9976419857 core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java 7c1aadf273aeca70c110f29ccf527c073250b2e7 core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 1bbf37d8d15c603666ddb081f83d515730740bbc core/src/main/java/org/apache/oozie/dependency/URIHandler.java bc947168558f1970cc2fd1c40b87a1c43ce860ee core/src/main/java/org/apache/oozie/util/WritableUtils.java 76a68953541125cab05bd05c94aa73b50e5363ff core/src/main/resources/oozie-default.xml faf37405757719554a5b6e3671d0c87723eb9959 core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java 1fe1b3adb5ceaa83985ef31278e49a3224cde0fb core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordInputCheckPush.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordinatorInputCheck.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/java/org/apache/oozie/coord/input/dependency/TestInputLogicParser.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/java/org/apache/oozie/coord/input/logic/TestCoordInputLogicPush.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/java/org/apache/oozie/coord/input/logic/TestCoordinatorInputLogic.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/java/org/apache/oozie/coord/input/logic/TestInputLogicParser.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/resources/coord-action-sla.xml f3f1bc09c51272a35d65edd7271599e7e8ba1ba4 core/src/test/resources/coord-inputlogic-combine.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/resources/coord-inputlogic-hcat.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/resources/coord-inputlogic-latest.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/resources/coord-inputlogic-range-latest.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/resources/coord-inputlogic-range.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 core/src/test/resources/coord-inputlogic.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 pom.xml a74ffab081defd15a1c5dbbf15355aa4e0455029 Diff: https://reviews.apache.org/r/38474/diff/ Testing ------- Thanks, Purshotam Shah