[ https://issues.apache.org/jira/browse/OOZIE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769501#comment-16769501 ]
Hadoop QA commented on OOZIE-3249: ---------------------------------- Testing JIRA OOZIE-3249 Cleaning local git workspace ---------------------------- {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} . {color:green}+1{color} the patch does not introduce any @author tags . {color:green}+1{color} the patch does not introduce any tabs . {color:green}+1{color} the patch does not introduce any trailing spaces . {color:green}+1{color} the patch does not introduce any star imports . {color:green}+1{color} the patch does not introduce any line longer than 132 . {color:red}-1{color} the patch does not add/modify any testcase {color:red}-1 RAT{color} . {color:red}-1{color} the patch seems to introduce 1 new RAT warning(s) {color:green}+1 JAVADOC{color} . {color:green}+1{color} Javadoc generation succeeded with the patch . {color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) . {color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s) {color:green}+1 COMPILE{color} . {color:green}+1{color} HEAD compiles . {color:green}+1{color} patch compiles . {color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/git]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [core]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} . {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . {color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} . Tests run: 3135 . {color:orange}Tests failed at first run:{color} TestPurgeXCommand#testPurgeableBundleUnpurgeableCoordinatorUnpurgebleWorkflowPurgeableSubWorkflow . For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} . {color:green}+1{color} distro tarball builds with the patch ---------------------------- {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/1001/ > [tools] Instrumentation log parser > ---------------------------------- > > Key: OOZIE-3249 > URL: https://issues.apache.org/jira/browse/OOZIE-3249 > Project: Oozie > Issue Type: Improvement > Components: tools > Affects Versions: 5.0.0 > Reporter: Andras Piros > Assignee: Andras Piros > Priority: Major > Attachments: OOZIE-3249.001.patch, > oozie-instrumentation-localhost.log.2018-05-09, > oozie-instrumentation-localhost.log.2018-05-09.out > > > Oozie instrumentation logs contain a lot of information, but are difficult to > parse, because per instrumentation log entry there is always one header line > in plain text format (containing timestamp), and multiple other lines in JSON > format (not containing timestamp). Those lines of course belong together. > {noformat} > 2018-05-02 02:48:13,426 INFO oozieinstrumentation:520 - USER[-] GROUP[-] > TOKEN[-] APP[-] JOB[-] ACTION[-] > { > ... > "counters" : { > ... > "callablequeue.executed" : { > "count" : 5954144 > }, > ... > "callablequeue.queued" : { > "count" : 10596129 > }, > ... > }, > ... > } > {noformat} > There should be a simple script in {{tools/bin}} that takes as parameters: > * input file name ({{-i}}), e.g. {{-i /path/to/oozie-instrumentation.log}} > * output file name ({{-o}}), e.g. {{-o > /path/to/oozie-instrumentation.log.out}} > * parameters to extract ({{-p}}) in the format of > {{path/to/json/value1,path/to/json/value2}}, in this case {{-p > counters/callablequeue.executed/count,counters/callablequeue.queued/count}} > The output file should contain in CSV format: > * a header line containing column names for > * one line per parsed input header / JSON lines, containing: > ** first cell is the minutes part of the timestamp > ** consecutive cells are parsed JSON values given each parameter to extract -- This message was sent by Atlassian JIRA (v7.6.3#76005)