> On Sept. 30, 2017, 4:41 a.m., Robert Kanter wrote: > > tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java > > Lines 151 (patched) > > <https://reviews.apache.org/r/62459/diff/7/?file=1837523#file1837523line151> > > > > Should we skip control types? That's probably fine for the start > > action, but what about a decision action? That has some useful info. > > Maybe we should still print these out, but with fewer fields or something > > (e.g. no need for a console url). > > Robert Kanter wrote: > Thinking about this more, I think we should include control types. It > will be helpful for following the flow of a workflow when there's decision > and fork nodes. Please update the patch to handle this. We don't need to > print out all of the fields, so you'll have to figure out which ones make > sense. > > Attila Sasvari wrote: > Sure, I will do so.
I believe only the decision node might interesting to see execution path, but I am not sure. ``info.txt`` already contains execution / transition info: ``` Action Id : "0000004-171015215418832-oozie-asas-W@ctrl" Name : "ctrl" Type : "switch" Status : "OK" Transition : "forking" Start Time : "2017-10-15 20:20:36 GMT" End Time : "2017-10-15 20:20:36 GMT" Error Code : null Error Message : null Console URL : "-" Tracker URI : "-" External Child Ids : null External Id : "-" External Status : "forking" Data : null Stats : null Credentials : null Retries : "0" User Retry Int : "10" User Retry Count : "0" User Retry Max : "0" ``` If I store resolved action defintion, it would not add too much information: ``` # cat 0000004-171015215418832-oozie-asas-W/resolved-actions/ctrl.xml <switch xmlns="uri:oozie:workflow:0.2"> <case to="forking">true</case> <default to="end" /> </switch>% ``` I created a workflow with a fork node and also with a decision node. I saw that resolved action config xml were empty for control nodes other than the decision node. ``` 0 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/:start:.xml 108 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/ctrl.xml 0 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/end.xml 0 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/forking.xml 0 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/join.xml 83834 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node.log 83843 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node2.log 47965 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_shell1.log 47965 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_shell2.log 2045 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/mr-node.xml 2045 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/mr-node2.xml 544 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/shell1.xml 544 10-15-17 22:44 /0000004-171015215418832-oozie-asas-W/resolved-actions/shell2.xml ``` Zero byte xml-s are not interesting. I will remove non-decision action types. Other thing: it would also good to add later an option to download the final job graph (thanks to OOZIE-2406 now we can download it as a Dot file too). If they have many action, it might help a lot when taking a quick look. Misc: Would it better to have a different pattern for the filename? For example: it might be good to see where the bundle was actually taken (i.e. include fqdn-hostname, there are existing diagnostic tools that use similar pattern). - Attila ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62459/#review186754 ----------------------------------------------------------- On Oct. 4, 2017, 2:18 p.m., Attila Sasvari wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62459/ > ----------------------------------------------------------- > > (Updated Oct. 4, 2017, 2:18 p.m.) > > > Review request for oozie. > > > Repository: oozie-git > > > Description > ------- > > A diagnostic tool that collects a bunch of job and other information from > Oozie in a zip file. > > > Diffs > ----- > > docs/src/site/twiki/DG_CommandLineTool.twiki > d4047671876dcc3279a2ec379bc1d003f5e6f1aa > pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c > tools/pom.xml 7306a14e7b237977be00f8fe28e34573540fd508 > tools/src/main/bin/oozie-diag-bundle-collector.sh PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java > PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/ArgParser.java PRE-CREATION > > tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCollectorDriver.java > PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCompressor.java > PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleEntryWriter.java > PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/DiagOozieClient.java > PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/MetricsCollector.java > PRE-CREATION > tools/src/main/java/org/apache/oozie/tools/diag/ServerInfoCollector.java > PRE-CREATION > tools/src/test/java/org/apache/oozie/tools/diag/TestAppInfoCollector.java > PRE-CREATION > tools/src/test/java/org/apache/oozie/tools/diag/TestArgParser.java > PRE-CREATION > tools/src/test/java/org/apache/oozie/tools/diag/TestMetricsCollector.java > PRE-CREATION > > tools/src/test/java/org/apache/oozie/tools/diag/TestServerInfoCollector.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/62459/diff/10/ > > > Testing > ------- > > - new unit tests: TestOozieDiagBundleCollector > - started Oozie with a pseudo hadoop cluster, submitted a couple workflows, > and executed the following commands: > -- ``bin/oozie-diag-bundle-collector.sh`` (usage info printed), > -- ``bin/oozie-diag-bundle-collector.sh -numworkflows 2000 -oozie > http://localhost:11000/oozie -output /tmp``, > -- ``bin/oozie-diag-bundle-collector.sh -jobs > 0000001-170918144116149-oozie-asas-W -oozie http://localhost:11000/oozie > -output .`` (verified zip the tool generated). > > > Thanks, > > Attila Sasvari > >