[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253472#comment-16253472 ] Attila Sasvari commented on OOZIE-2296: --- Thanks for the reviews and recommendations, committed to master. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch, OOZIE-2296-017.patch, > OOZIE-2296-018.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253397#comment-16253397 ] Peter Cseh commented on OOZIE-2296: --- +1! This would be a great addition to Oozie. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch, OOZIE-2296-017.patch, > OOZIE-2296-018.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253216#comment-16253216 ] Attila Sasvari commented on OOZIE-2296: --- [~andras.piros], [~gezapeti], [~pbacsko] do you have any other comments or recommendation? Do you see the need of additional followup JIRA-s? > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch, OOZIE-2296-017.patch, > OOZIE-2296-018.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251689#comment-16251689 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [server]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 0 .Tests rerun: 0 .Tests failed at first run: org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.command.coord.TestCoordSubmitXCommand,org.apache.oozie.util.TestMetricsInstrumentation,org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand,org.apache.oozie.service.TestConfigurationService, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/216/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch, OOZIE-2296-017.patch, > OOZIE-2296-018.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249697#comment-16249697 ] Attila Sasvari commented on OOZIE-2296: --- - {{testImportInvalidDataLeavesTablesEmpty}} failure is related to test execution order, see OOZIE-3125 - {{testJMXInstrumentation}} is tracked by OOZIE-3105 > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch, OOZIE-2296-017.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240161#comment-16240161 ] Attila Sasvari commented on OOZIE-2296: --- Executed tests locally: {code} Failed tests: testImportInvalidDataLeavesTablesEmpty(org.apache.oozie.tools.TestDBLoadDump): import should have been ended prematurely Failed tests: testMaterizationLookup(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand): expected: but was: Failed tests: testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation): Could not fetch metric {code} I will investigate failure of {{testJMXInstrumentation}} and {{testImportInvalidDataLeavesTablesEmpty}} soon (these should not be related to the patch in my opinion). {{testMaterizationLookup}} is known to be flaky. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch, OOZIE-2296-017.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16239348#comment-16239348 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2063 .Tests failed: 9 .Tests errors: 3 .The patch failed the following testcases: . testImportTablesOverflowBatchSize(org.apache.oozie.tools.TestDBLoadDump) . testImportInvalidDataLeavesTablesEmpty(org.apache.oozie.tools.TestDBLoadDump) . testImportToNonEmptyTablesCausesPrematureExit(org.apache.oozie.tools.TestDBLoadDump) . testImportedDBIsExportedCorrectly(org.apache.oozie.tools.TestDBLoadDump) . testMaterizationLookup(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand) . testMain(org.apache.oozie.action.hadoop.TestHiveMain) . testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain) . testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain) . testPigScript(org.apache.oozie.action.hadoop.TestPigMain) .Tests failing with errors: . testAddXIncludeFromReader(org.apache.oozie.util.TestXConfiguration) . testAddXIncludeFromStream(org.apache.oozie.util.TestXConfiguration) . testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation) {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/174/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch,
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16239322#comment-16239322 ] Attila Sasvari commented on OOZIE-2296: --- Generated bundle is as follows: {code} 2741 11-05-17 00:28 /000-171105002642017-oozie-asas-W/info.txt 307 11-05-17 00:28 /000-171105002642017-oozie-asas-W/job.properties 8477 11-05-17 00:28 /000-171105002642017-oozie-asas-W/log.txt 0 11-05-17 00:28 /000-171105002642017-oozie-asas-W/resolved-actions/ 0 11-05-17 00:28 /000-171105002642017-oozie-asas-W/resolved-actions/:start:.xml 0 11-05-17 00:28 /000-171105002642017-oozie-asas-W/resolved-actions/end.xml 83709 11-05-17 00:28 /000-171105002642017-oozie-asas-W/resolved-actions/launcher_mr-node.log 2045 11-05-17 00:28 /000-171105002642017-oozie-asas-W/resolved-actions/mr-node.xml 2559 11-05-17 00:28 /000-171105002642017-oozie-asas-W/workflow.xml 66586 11-05-17 00:28 /effective-oozie-site.xml 23946 11-05-17 00:28 /instrumentation.txt 37499 11-05-17 00:28 /java-sys-props.txt 3635 11-05-17 00:28 /os-env-vars.txt 279 11-05-17 00:28 /queue-dump.txt 38423 11-05-17 00:28 /sharelib.txt 100476 11-05-17 00:28 /thread-dump.html {code} - {{/000-171105002642017-oozie-asas-W/info.txt}} contains state transitions - {{/000-171105002642017-oozie-asas-W/resolved-actions/mr-node.xml}} contains the resolved workflow action "mr-node" (e.g. EL expressions evaluated) - {{/000-171105002642017-oozie-asas-W/resolved-actions/launcher_mr-node.log}} is the launcher for the action (here I had a simple workflow with 1 MR action) - {{/000-171105002642017-oozie-asas-W/workflow.xml}} is the user's workflow (without EL evaluation) - {{/000-171105002642017-oozie-asas-W/log.txt}} contain the server logs of for the {{000-171105002642017-oozie-asas-W}} workflow > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222118#comment-16222118 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2056 .Tests failed: 0 .Tests errors: 1 .The patch failed the following testcases: . .Tests failing with errors: . testStoreLastWorkflows(org.apache.oozie.tools.diag.TestAppInfoCollector) {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/138/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209237#comment-16209237 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/121/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch, OOZIE-2296-016.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #-
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206831#comment-16206831 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [examples]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/118/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206286#comment-16206286 ] Robert Kanter commented on OOZIE-2296: -- You mentioned in one of the comments on RB the idea of getting the DAG image, especially now that OOZIE-2406 is done and we can get a dot file now too. This is a great idea! However, we should probably wait for OOZIE-3059 first. It's to add a CLI command to get the image. While you could have the tool use the REST API directly to do that, that's going to be 90% overlap with the code needed for OOZIE-3059, so you may as well do OOZIE-3059 instead. I'm fine with getting the image being a followup JIRA if you don't want to block the rest of this on OOZIE-3059. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206028#comment-16206028 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [examples]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/116/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch, > OOZIE-2296-015.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205683#comment-16205683 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/114/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs'
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205353#comment-16205353 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/113/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch, OOZIE-2296-014.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs'
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196376#comment-16196376 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [3] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [3] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 2049 .Tests rerun: 98 .Tests failed at first run: org.apache.oozie.service.TestCoordMaterializeTriggerService,org.apache.oozie.util.TestInstrumentation,org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestJavaActionExecutor, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/87/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch, OOZIE-2296-013.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193058#comment-16193058 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 2 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [3] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:red}-1{color} There are [3] new bugs found below threshold in [tools] that must be fixed. . You can find the FindBugs diff here (look for the red and orange ones): tools/findbugs-new.html . The most important FindBugs errors are: . At AppInfoCollector.java:[line 467]: Argument of type String can't be handled by format specifier %d in org.apache.oozie.tools.diag.AppInfoCollector.storeLastCoordinators(File, int, int) . At AppInfoCollector.java:[line 451]: Argument of type String can't be handled by format specifier %d in org.apache.oozie.tools.diag.AppInfoCollector.storeLastWorkflows(File, int, int) . At AppInfoCollector.java:[line 483]: Argument of type String can't be handled by format specifier %d in org.apache.oozie.tools.diag.AppInfoCollector.stroreLastBundles(File, int, int) . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 2049 .Tests rerun: 71 .Tests failed at first run: org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/80/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192868#comment-16192868 ] Peter Cseh commented on OOZIE-2296: --- I think the leftover comments are small enough to commit the latest version and fix minor things down the road as they come up. I've retriggered the pre-commit hook for the latest patch as the test failures don't seem related. +1 (pending Jenkins) > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, > OOZIE-2296-012.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191534#comment-16191534 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 2 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [3] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:red}-1{color} There are [3] new bugs found below threshold in [tools] that must be fixed. . You can find the FindBugs diff here (look for the red and orange ones): tools/findbugs-new.html . The most important FindBugs errors are: . At AppInfoCollector.java:[line 467]: Argument of type String can't be handled by format specifier %d in org.apache.oozie.tools.diag.AppInfoCollector.storeLastCoordinators(File, int, int) . At AppInfoCollector.java:[line 451]: Argument of type String can't be handled by format specifier %d in org.apache.oozie.tools.diag.AppInfoCollector.storeLastWorkflows(File, int, int) . At AppInfoCollector.java:[line 483]: Argument of type String can't be handled by format specifier %d in org.apache.oozie.tools.diag.AppInfoCollector.stroreLastBundles(File, int, int) . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2049 .Tests failed: 8 .Tests errors: 2 .The patch failed the following testcases: . testImportTablesOverflowBatchSize(org.apache.oozie.tools.TestDBLoadDump) . testImportInvalidDataLeavesTablesEmpty(org.apache.oozie.tools.TestDBLoadDump) . testImportToNonEmptyTablesCausesPrematureExit(org.apache.oozie.tools.TestDBLoadDump) . testImportedDBIsExportedCorrectly(org.apache.oozie.tools.TestDBLoadDump) . testMain(org.apache.oozie.action.hadoop.TestHiveMain) . testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain) . testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain) . testPigScript(org.apache.oozie.action.hadoop.TestPigMain) .Tests failing with errors: . testAddXIncludeFromReader(org.apache.oozie.util.TestXConfiguration) . testAddXIncludeFromStream(org.apache.oozie.util.TestXConfiguration) {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/76/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature >
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188264#comment-16188264 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [examples]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/65/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185490#comment-16185490 ] Andras Piros commented on OOZIE-2296: - [~asasvari] left some comments on RB. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183027#comment-16183027 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:red}-1 RAT{color} .{color:red}-1{color} the patch seems to introduce 2 new RAT warning(s) {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [4] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:red}-1{color} There are [4] new bugs found below threshold in [tools] that must be fixed. . You can find the FindBugs diff here (look for the red and orange ones): tools/findbugs-new.html . The most important FindBugs errors are: . At AppInfoCollector.java:[line 336]: File(...) reads a file whose location might be specified by user input . At AppInfoCollector.java:[line 487]: At AppInfoCollector.java:[line 468] . At BundleCollectorDriver.java:[line 118]: At BundleCollectorDriver.java:[line 68] . At AppInfoCollector.java:[line 242]: File(...) reads a file whose location might be specified by user input . At AppInfoCollector.java:[line 452]: At AppInfoCollector.java:[line 355] . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 2047 .Tests rerun: 69 .Tests failed at first run: org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/53/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182646#comment-16182646 ] Attila Sasvari commented on OOZIE-2296: --- Findbugs warning with reason "... reads a file whose location might be specified by user input" is expected as we allow the user to specify the directory where the bundle zip will be stored. [~pbacsko], [~gezapeti], [~andras.piros] can you take a look at the latest patch? Retrieving Oozie launcher logs is currently done with the {{LogsCLI}} client tool from Yarn. If we lose connection to Resource Manager when trying to fetch launcher logs for a job and the connection cannot be re-established in 30 seconds, we will give up trying, display stack trace and some error message and continue to collect diagnostic information on the rest of the applications. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch, OOZIE-2296-010.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181733#comment-16181733 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 2 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 4 testcase(s) {color:red}-1 RAT{color} .{color:red}-1{color} the patch seems to introduce 2 new RAT warning(s) {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [4] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:red}-1{color} There are [4] new bugs found below threshold in [tools] that must be fixed. . You can find the FindBugs diff here (look for the red and orange ones): tools/findbugs-new.html . The most important FindBugs errors are: . At AppInfoCollector.java:[line 337]: File(...) reads a file whose location might be specified by user input . At AppInfoCollector.java:[line 487]: At AppInfoCollector.java:[line 468] . At BundleCollectorDriver.java:[line 118]: At BundleCollectorDriver.java:[line 68] . At AppInfoCollector.java:[line 243]: File(...) reads a file whose location might be specified by user input . At AppInfoCollector.java:[line 452]: At AppInfoCollector.java:[line 356] . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2049 .Tests failed: 1 .Tests errors: 0 .The patch failed the following testcases: . instrumentionCanBeRetrieved(org.apache.oozie.tools.diag.TestMetricsCollector) .Tests failing with errors: . {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/52/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, > OOZIE-2296-009.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178921#comment-16178921 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 1 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [1] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [1] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 2037 .Tests rerun: 129 .Tests failed at first run: org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.TestDagEngine,org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/49/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #-
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178727#comment-16178727 ] Attila Sasvari commented on OOZIE-2296: --- [~pbacsko] many thanks for the reivew. I fixed some of the issues mentioned. I will also try to refactor the code (extracting classes mostly) to make it more testable. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177796#comment-16177796 ] Peter Bacsko commented on OOZIE-2296: - I added some comments to the review. I would possibly add more, depending on how we agree on the refactoring/restructuring part. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176575#comment-16176575 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 1 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:orange}0{color} There are [1] new bugs found in total that would be nice to have fixed. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:orange}0{color} There are [1] new bugs found in [tools] that would be nice to have fixed. . You can find the FindBugs diff here: tools/findbugs-new.html . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 2037 .Tests rerun: 94 .Tests failed at first run: org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand,org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.command.TestXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/45/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch, OOZIE-2296-005.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175307#comment-16175307 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 1 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 1770 .Tests rerun: 10 .Tests failed at first run: org.apache.oozie.TestSLAEventBean,org.apache.oozie.TestDagELFunctions,org.apache.oozie.TestDagEngine,org.apache.oozie.TestCoordinatorEngineStreamLog, {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/43/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, > OOZIE-2296-004.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174658#comment-16174658 ] Hadoop QA commented on OOZIE-2296: -- Testing JIRA OOZIE-2296 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 3 line(s) longer than 132 characters .{color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings .{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [6] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:red}-1{color} There are [6] new bugs found below threshold in [tools] that must be fixed, listing only the first [5] ones. . You can find the FindBugs diff here (look for the red and orange ones): tools/findbugs-new.html . The top [5] most important FindBugs errors are: . At OozieDiagBundleCollector.java:[line 178]: org.apache.oozie.tools.OozieDiagBundleCollector.parseCommandLineArgs(String[]) invokes System.exit(...), which shuts down the entire virtual machine . Method invoked at OozieDiagBundleCollector.java:[line 740]: Null passed for non-null parameter of new java.io.File(File, String) in org.apache.oozie.tools.OozieDiagBundleCollector.getBundleJob(OozieClient, File, String, int) . File(...) reads a file whose location might be specified by user input: Known null at OozieDiagBundleCollector.java:[line 734] . At OozieDiagBundleCollector.java:[line 145]: At OozieDiagBundleCollector.java:[line 740] . At OozieDiagBundleCollector.java:[line 200]: At OozieDiagBundleCollector.java:[line 156] {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} - patch does not compile, cannot run testcases {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/42/ > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172050#comment-16172050 ] Robert Kanter commented on OOZIE-2296: -- Sure. > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Robert Kanter > Attachments: OOZIE-2296-002.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool
[ https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171750#comment-16171750 ] Attila Sasvari commented on OOZIE-2296: --- [~rkanter] Do you mind if I take over this task? > Add an Oozie diagnostic bundle tool > --- > > Key: OOZIE-2296 > URL: https://issues.apache.org/jira/browse/OOZIE-2296 > Project: Oozie > Issue Type: New Feature > Components: tools >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Robert Kanter > Attachments: OOZIE-2296-002.patch > > > To help with our support cases, I've built a tool that collects a bunch of > job and other information from Oozie that throws it all in a tarball. The > idea is that the user can just click a button in Cloudera Manager, and it > will run this tool. > This tool could be useful for others as an easy way to get information out of > Oozie, so I thought I'd contribute it here. It's built as a "tool" (so it > sits next to the sharelib and database tools), and simply uses the Oozie > client for getting pretty much everything, so it doesn't require anything > special. > Here's the information that it can get: > # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist > }} > # Oozie Server's resolved loaded configuration (from admin endpoint) > # Other admin commands output (queue dump, env vars, etc) > # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp) > # Details from last n jobs and/or specific list of jobs > #- job.properties contents > #- XML definition > #- verbose status for each job and each action etc > #- Oozie logs > #- Unfortunately, we can't get the launcher jobs' logs from Hadoop > # Metrics/Instrumentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)