date:20171005

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

2017-10-05 Thread Robert Kanter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62352/#review187224
---


Fix it, then Ship it!




Looks great!  Just some trivial things.


core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java
Lines 24 (patched)


Let's not use * imports



core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java
Lines 51 (patched)


Extra space before @param



core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java
Lines 67 (patched)


This isn't always a PNG stream, right?



core/src/main/java/org/apache/oozie/util/graph/GraphvizRenderer.java
Lines 39 (patched)


Let's not use * imports



core/src/main/java/org/apache/oozie/util/graph/GraphvizRenderer.java
Lines 53 (patched)


oozie.graphviz.timeout.seconds should be a constant.  No need to 
put a default value here because it's in oozie-default.xml and we want to keep 
only one source of truth on those.
ConfigurationService.getLong(OOZIE_GRAPHVIZ_TIMEOUT_SECONDS);



core/src/test/java/org/apache/oozie/servlet/TestV1JobServlet.java
Lines 394 (patched)


// format=svg -> OutputFormat.SVG


- Robert Kanter


On Oct. 5, 2017, 11:02 a.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62352/
> ---
> 
> (Updated Oct. 5, 2017, 11:02 a.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2406 Completely rewrite GraphGenerator code
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 74843dc9a3cbb8cef38e97f81e34731f191d9aab 
>   core/pom.xml 6f9adb66af9344ac7d2212cdc31aa203ec06c286 
>   core/src/main/java/org/apache/oozie/servlet/JsonRestServlet.java 
> 059d3cf6dc251b49940af29d82cbdd817043a176 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 
> 10812c675ebc0cc4aefca9f4a85ef2fc26d143e1 
>   core/src/main/java/org/apache/oozie/util/GraphGenerator.java 
> 6ded2c6dc15c9e8453ff800407ff0324be185f41 
>   core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/GraphRenderer.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/GraphvizRenderer.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/OutputFormat.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/WorkflowActionNode.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/WorkflowGraphHandler.java 
> PRE-CREATION 
>   core/src/main/resources/oozie-default.xml 
> 2389b993b7b1769521d6dd0c7386e99e7516210a 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobServlet.java 
> ee9ab556c9b6930c406ca5dcd54065cbbebb86c9 
>   core/src/test/java/org/apache/oozie/util/TestGraphGenerator.java 
> 002e925b57cd830ea6d83a87cea4383165116b80 
>   core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java 
> PRE-CREATION 
>   core/src/test/resources/graph-workflow-decision-fork-join.xml PRE-CREATION 
>   core/src/test/resources/graph-workflow-simple.xml PRE-CREATION 
>   core/src/test/resources/graphWF.xml 
> 6a7b0427a9951835a7533a04b66258ded369d5bf 
>   core/src/test/resources/graphWF_26_actions.xml 
> a091be0f3559ede195ccc3339adee4478a8da8c0 
>   core/src/test/resources/invalidGraphWF.xml  
>   docs/src/site/twiki/WebServicesAPI.twiki 
> ef3e60242512decd48beb3d8c9ac747b7d128eda 
>   pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c 
>   webapp/src/main/webapp/oozie-console.js 
> 72c8a198a4ffe60f74a9f700831f65efcb3066c4 
> 
> 
> Diff: https://reviews.apache.org/r/62352/diff/7/
> 
> 
> Testing
> ---
> 
> `TestGraphGenerator`, `TestV1JobServlet`
> 
> 
> Thanks,
> 
> András Piros
> 
>

Re: Review Request 62459: OOZIE-2296: Add an Oozie diagnostic bundle tool

2017-10-05 Thread Robert Kanter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62459/#review187223
---




docs/src/site/twiki/DG_CommandLineTool.twiki
Lines 1989 (patched)


Shouldn't this "Done" show up on the previous line like the rest of them?



tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
Lines 173-174 (patched)


LogAggregationUtils is marked @Private so we shouldn't be 
using it.  Hadoop can change things incompatibly here.



tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
Lines 180 (patched)


var20 is a funny name for a variable :)



tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
Lines 241 (patched)


storeOozieLauncherLog should be called per Action, not per Job.  
Launcher logs are an Action-level concept.

Here, you're just picking the first action's external id to use for the 
job, which may not even exist (e.g. the first action is a decision node or an 
hdfs node.



tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
Lines 242 (patched)


We should get this from the WorkflowJob.  The log reading code 
computes a path based on the user who ran the job, which is likely to be 
different from the user running this tool.


- Robert Kanter


On Oct. 4, 2017, 2:18 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62459/
> ---
> 
> (Updated Oct. 4, 2017, 2:18 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> A diagnostic tool that collects a bunch of job and other information from 
> Oozie in a zip file.
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 
> d4047671876dcc3279a2ec379bc1d003f5e6f1aa 
>   pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c 
>   tools/pom.xml 7306a14e7b237977be00f8fe28e34573540fd508 
>   tools/src/main/bin/oozie-diag-bundle-collector.sh PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/ArgParser.java PRE-CREATION 
>   
> tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCollectorDriver.java
>  PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCompressor.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleEntryWriter.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/DiagOozieClient.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/MetricsCollector.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/ServerInfoCollector.java 
> PRE-CREATION 
>   tools/src/test/java/org/apache/oozie/tools/diag/TestAppInfoCollector.java 
> PRE-CREATION 
>   tools/src/test/java/org/apache/oozie/tools/diag/TestArgParser.java 
> PRE-CREATION 
>   tools/src/test/java/org/apache/oozie/tools/diag/TestMetricsCollector.java 
> PRE-CREATION 
>   
> tools/src/test/java/org/apache/oozie/tools/diag/TestServerInfoCollector.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62459/diff/10/
> 
> 
> Testing
> ---
> 
> - new unit tests: TestOozieDiagBundleCollector
> - started Oozie with a pseudo hadoop cluster, submitted a couple workflows, 
> and executed the following commands: 
> -- ``bin/oozie-diag-bundle-collector.sh`` (usage info printed),
> -- ``bin/oozie-diag-bundle-collector.sh  -numworkflows 2000 -oozie 
> http://localhost:11000/oozie -output /tmp``, 
> -- ``bin/oozie-diag-bundle-collector.sh  -jobs 
> 001-170918144116149-oozie-asas-W -oozie http://localhost:11000/oozie 
> -output .`` (verified zip the tool generated).
> 
> 
> Thanks,
> 
> Attila Sasvari
> 
>

Re: Review Request 62459: OOZIE-2296: Add an Oozie diagnostic bundle tool

2017-10-05 Thread Robert Kanter



> On Sept. 30, 2017, 4:41 a.m., Robert Kanter wrote:
> > tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
> > Lines 151 (patched)
> > 
> >
> > Should we skip control types?  That's probably fine for the start 
> > action, but what about a decision action?  That has some useful info.  
> > Maybe we should still print these out, but with fewer fields or something 
> > (e.g. no need for a console url).

Thinking about this more, I think we should include control types.  It will be 
helpful for following the flow of a workflow when there's decision and fork 
nodes.  Please update the patch to handle this.  We don't need to print out all 
of the fields, so you'll have to figure out which ones make sense.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62459/#review186754
---


On Oct. 4, 2017, 2:18 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62459/
> ---
> 
> (Updated Oct. 4, 2017, 2:18 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> A diagnostic tool that collects a bunch of job and other information from 
> Oozie in a zip file.
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 
> d4047671876dcc3279a2ec379bc1d003f5e6f1aa 
>   pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c 
>   tools/pom.xml 7306a14e7b237977be00f8fe28e34573540fd508 
>   tools/src/main/bin/oozie-diag-bundle-collector.sh PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/ArgParser.java PRE-CREATION 
>   
> tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCollectorDriver.java
>  PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCompressor.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleEntryWriter.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/DiagOozieClient.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/MetricsCollector.java 
> PRE-CREATION 
>   tools/src/main/java/org/apache/oozie/tools/diag/ServerInfoCollector.java 
> PRE-CREATION 
>   tools/src/test/java/org/apache/oozie/tools/diag/TestAppInfoCollector.java 
> PRE-CREATION 
>   tools/src/test/java/org/apache/oozie/tools/diag/TestArgParser.java 
> PRE-CREATION 
>   tools/src/test/java/org/apache/oozie/tools/diag/TestMetricsCollector.java 
> PRE-CREATION 
>   
> tools/src/test/java/org/apache/oozie/tools/diag/TestServerInfoCollector.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62459/diff/10/
> 
> 
> Testing
> ---
> 
> - new unit tests: TestOozieDiagBundleCollector
> - started Oozie with a pseudo hadoop cluster, submitted a couple workflows, 
> and executed the following commands: 
> -- ``bin/oozie-diag-bundle-collector.sh`` (usage info printed),
> -- ``bin/oozie-diag-bundle-collector.sh  -numworkflows 2000 -oozie 
> http://localhost:11000/oozie -output /tmp``, 
> -- ``bin/oozie-diag-bundle-collector.sh  -jobs 
> 001-170918144116149-oozie-asas-W -oozie http://localhost:11000/oozie 
> -output .`` (verified zip the tool generated).
> 
> 
> Thanks,
> 
> Attila Sasvari
> 
>

[jira] [Resolved] (OOZIE-1723) Setting 'use.new.api=true' gives compatibility error even with using new 'mapreduce.*' hadoop api

2017-10-05 Thread Rohini Palaniswamy (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy resolved OOZIE-1723.
---
Resolution: Not A Problem

https://cwiki.apache.org/confluence/display/OOZIE/Map+Reduce+Cookbook

To run MapReduce jobs written using the new API in Oozie, below are the changes 
you need to make in workflow.xml.

#   change mapred.mapper.class to mapreduce.map.class
#   change mapred.reducer.class to mapreduce.reduce.class
#add mapred.output.key.class
#add mapred.output.value.class
#and, include the following property into MR action configuration
{code}

mapred.reducer.new-api
true
  
  
mapred.mapper.new-api
true

{code}

> Setting 'use.new.api=true' gives compatibility error even with using new 
> 'mapreduce.*' hadoop api
> -
>
> Key: OOZIE-1723
> URL: https://issues.apache.org/jira/browse/OOZIE-1723
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk, 4.0.0
>Reporter: Mona Chitnis
>  Labels: api
>
> Error encountered - JA009: mapred.mapper.class is incompatible with new map 
> API mode.
> complete stack trace:
> {code}
> org.apache.oozie.action.ActionExecutorException: JA009: mapred.mapper.class 
> is incompatible with new map API mode.
>   at 
> org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:396)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:871)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1025)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:217)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:62)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:280)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.IOException: mapred.mapper.class is incompatible with new 
> map API mode.
>   at org.apache.hadoop.mapreduce.Job.ensureNotSet(Job.java:1122)
>   at org.apache.hadoop.mapreduce.Job.setUseNewAPI(Job.java:1149)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1211)
>   at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments

2017-10-05 Thread Peter Cseh (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193757#comment-16193757
 ] 

Peter Cseh commented on OOZIE-3078:
---

I think we should go ahead and make every public function's behavior the same.
After your patch every path going into [this 
call|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L120]
 will throw an exception while this call will return "null" as the unmasked 
value.

I think we should remove the precondition checking and say that we won't mask 
the value even if the [key is null 
here|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L145]
 so we won't break workflows with NPEs.


> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch
>
>
> If 
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
>  is called with a {{null}} argument, a NPE will be thrown. It should return 
> false.
> Note: [SparkMain | 
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
>  uses this method.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (OOZIE-3078) PasswordMasker throws NPE with null arguments

2017-10-05 Thread Peter Cseh (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193757#comment-16193757
 ] 

Peter Cseh edited comment on OOZIE-3078 at 10/5/17 9:32 PM:


I think we should go ahead and make every public function's behavior the same.
After your patch every path going into [this 
call|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L120]
 will throw an exception while the path you've changed will return "null" as 
the unmasked value.

I think we should remove the precondition checking and say that we won't mask 
the value even if the [key is null 
here|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L145]
 so we won't break workflows with NPEs.



was (Author: gezapeti):
I think we should go ahead and make every public function's behavior the same.
After your patch every path going into [this 
call|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L120]
 will throw an exception while this call will return "null" as the unmasked 
value.

I think we should remove the precondition checking and say that we won't mask 
the value even if the [key is null 
here|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L145]
 so we won't break workflows with NPEs.


> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch
>
>
> If 
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
>  is called with a {{null}} argument, a NPE will be thrown. It should return 
> false.
> Note: [SparkMain | 
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
>  uses this method.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments

2017-10-05 Thread Attila Sasvari (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193569#comment-16193569
 ] 

Attila Sasvari commented on OOZIE-3078:
---

[~andras.piros] thanks for the review. [~pbacsko] thanks for the fix. Do you 
have other comments? [~gezapeti] can you also take a look?

Other comments:
- Right now the 
[mask()|https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L119]
 method throws an NPE if either key or value is {{null}}. It will cause a 
workflow to fail... It might be better to just ignore those elements (i.e. do 
not try to mask null-s). {{mask(String,String)}} is called from the public 
method {{mask(Map unmasked)}} (used by the Instrumentation 
class).

> PasswordMasker throws NPE with null arguments
> -
>
> Key: OOZIE-3078
> URL: https://issues.apache.org/jira/browse/OOZIE-3078
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3078-01.patch
>
>
> If 
> [maskPasswordsIfNecessary|https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PasswordMasker.java#L137]
>  is called with a {{null}} argument, a NPE will be thrown. It should return 
> false.
> Note: [SparkMain | 
> https://github.com/apache/oozie/blob/71d3ddc31cfb983dc83c51882f99c4429253230a/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L88]
>  uses this method.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null

2017-10-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193518#comment-16193518
 ] 

Hadoop QA commented on OOZIE-3079:
--

Testing JIRA OOZIE-3079

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2035
.Tests failed: 0
.Tests errors: 9

.The patch failed the following testcases:

.  

.Tests failing with errors:
.  testJobMethods(org.apache.oozie.TestLocalOozieClientCoord)
.  testCoordJobsGet(org.apache.oozie.command.coord.TestCoordJobsXCommand)
.  
testGetJobInfoForWrongTimeFormat(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
.  
testGetJobInfoForText(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
.  
testCoordJobGet(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
.  
testGetJobInfoForEndCreatedTime(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
.  
testCoordGetJobsSortBy(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
.  
testGetJobInfoForStartCreatedTime(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
.  testEngine(org.apache.oozie.TestCoordinatorEngine)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/81/

> Filtering coordinators returns bundle id as null
> 
>
> Key: OOZIE-3079
> URL: https://issues.apache.org/jira/browse/OOZIE-3079
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-3079-1.patch, OOZIE-3079-2.patch
>
>
> If we filter coordinator jobs, the json reply shows (parent) bundle id as 
> null even though coordinator was started from a bundle.
> Example query - 
> http://localhost:11000/oozie/v1/jobs?filter=user=saley=coordinator



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-3079 PreCommit Build #81

2017-10-05 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-3079
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/81/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 1.59 MB...]
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2035
Tests failed: 0
Tests errors: 9

The patch failed the following testcases:

  

Tests failing with errors:
  testJobMethods(org.apache.oozie.TestLocalOozieClientCoord)
  testCoordJobsGet(org.apache.oozie.command.coord.TestCoordJobsXCommand)
  
testGetJobInfoForWrongTimeFormat(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
  
testGetJobInfoForText(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
  
testCoordJobGet(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
  
testGetJobInfoForEndCreatedTime(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
  
testCoordGetJobsSortBy(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
  
testGetJobInfoForStartCreatedTime(org.apache.oozie.executor.jpa.TestCoordJobInfoGetJPAExecutor)
  testEngine(org.apache.oozie.TestCoordinatorEngine)

+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/81/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  5 3706k5  191k0 0   272k  0  0:00:13 --:--:--  0:00:13  
272k100 3706k  100 3706k0 0  4576k  0 --:--:-- --:--:-- --:--:-- 
32.0M
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3079
Archiving artifacts
[Fast Archiver] Compressed 1.80 MB of artifacts by 80.1% relative to #75
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
9 tests failed.
FAILED:  org.apache.oozie.TestCoordinatorEngine.testEngine

Error Message:
E0603: SQL error in operation, 16

Stack Trace:
org.apache.oozie.CoordinatorEngineException: E0603: SQL error in operation, 16
at 
org.apache.oozie.CoordinatorEngine.getCoordJobs(CoordinatorEngine.java:601)
at 
org.apache.oozie.TestCoordinatorEngine._testGetJobs(TestCoordinatorEngine.java:344)
at 
org.apache.oozie.TestCoordinatorEngine.testEngine(TestCoordinatorEngine.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at junit.framework.TestCase.runBare(TestCase.java:141)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:129)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at

[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null

2017-10-05 Thread Rohini Palaniswamy (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193402#comment-16193402
 ] 

Rohini Palaniswamy commented on OOZIE-3079:
---

+1

> Filtering coordinators returns bundle id as null
> 
>
> Key: OOZIE-3079
> URL: https://issues.apache.org/jira/browse/OOZIE-3079
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-3079-1.patch, OOZIE-3079-2.patch
>
>
> If we filter coordinator jobs, the json reply shows (parent) bundle id as 
> null even though coordinator was started from a bundle.
> Example query - 
> http://localhost:11000/oozie/v1/jobs?filter=user=saley=coordinator



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0

2017-10-05 Thread Artem Ervits (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193235#comment-16193235
 ] 

Artem Ervits commented on OOZIE-3071:
-

[~gezapeti] I checked spark 1.6 branch and I see it is compiled with 3.5 just 
as Spark 2.2 branch. Compiling with either spark version on 3.6 works fine. Is 
there anything you would propose to do in terms of pulling out the dep out of 
main pom for example?

> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than 
> Spark 2.2.0
> 
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
>  Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5. 
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: 
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at 
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at 
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at 
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at 
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at 
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at 
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at 
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0

2017-10-05 Thread Peter Cseh (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193223#comment-16193223
 ] 

Peter Cseh commented on OOZIE-3071:
---

My understanding that this dependency caused a runtime issue when Oozie was 
compiled and run with Spark2. My guess is that bumping this version could cause 
runtime issues with Spark1.6.3.


> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than 
> Spark 2.2.0
> 
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
>  Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5. 
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: 
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at 
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at 
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at 
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at 
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at 
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at 
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at 
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3079) Filtering coordinators returns bundle id as null

2017-10-05 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-3079:
---
Attachment: OOZIE-3079-2.patch

Uploading same patch again.

> Filtering coordinators returns bundle id as null
> 
>
> Key: OOZIE-3079
> URL: https://issues.apache.org/jira/browse/OOZIE-3079
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-3079-1.patch, OOZIE-3079-2.patch
>
>
> If we filter coordinator jobs, the json reply shows (parent) bundle id as 
> null even though coordinator was started from a bundle.
> Example query - 
> http://localhost:11000/oozie/v1/jobs?filter=user=saley=coordinator



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0

2017-10-05 Thread Artem Ervits (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193074#comment-16193074
 ] 

Artem Ervits commented on OOZIE-3071:
-

[~gezapeti] I was able to compile my change with Spark 1.6.3, unless I'm 
missing something, please let me know and I'll pick up the jira to work further 
on.

{noformat}
bin/mkdistro.sh -DjavaVersion=1.8 -fae -Dmaven.test.failure.ignore=true 
-Dspark.version=1.6.3 -DskipTests=true
{noformat}

> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than 
> Spark 2.2.0
> 
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
>  Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5. 
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: 
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at 
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at 
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at 
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at 
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at 
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at 
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at 
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool

2017-10-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193058#comment-16193058
 ] 

Hadoop QA commented on OOZIE-2296:
--

Testing JIRA OOZIE-2296

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 2 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1{color} There are [3] new bugs found below threshold in total that 
must be fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:red}-1{color} There are [3] new bugs found below threshold in [tools] 
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones): 
tools/findbugs-new.html
. The most important FindBugs errors are:
. At AppInfoCollector.java:[line 467]: Argument of type String can't be handled 
by format specifier %d in 
org.apache.oozie.tools.diag.AppInfoCollector.storeLastCoordinators(File, int, 
int)
. At AppInfoCollector.java:[line 451]: Argument of type String can't be handled 
by format specifier %d in 
org.apache.oozie.tools.diag.AppInfoCollector.storeLastWorkflows(File, int, int)
. At AppInfoCollector.java:[line 483]: Argument of type String can't be handled 
by format specifier %d in 
org.apache.oozie.tools.diag.AppInfoCollector.stroreLastBundles(File, int, int)
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2049
.Tests rerun: 71
.Tests failed at first run: 
org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/80/

> Add an Oozie diagnostic bundle tool
> ---
>
> Key: OOZIE-2296
> URL: https://issues.apache.org/jira/browse/OOZIE-2296
> Project: Oozie
>  Issue Type: New Feature
>  Components: tools
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
> Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, 
> OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, 
> OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, 
> OOZIE-2296-012.patch
>
>
> To help with our support cases, I've built a tool that collects a bunch of 
> job and other information from Oozie that throws it all in a tarball.  The 
> idea is that the user can just click a button in Cloudera Manager, and it 
> will run this tool.  
> This tool could be useful for others as an easy way to get information out of 
> Oozie, so I thought

Failed: OOZIE-2296 PreCommit Build #80

2017-10-05 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-2296
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/80/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 1.59 MB...]
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [3] new bugs found below threshold in total that must be fixed.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [examples].
 -1 There are [3] new bugs found below threshold in [tools] that must be fixed.
 You can find the FindBugs diff here (look for the red and orange ones): 
tools/findbugs-new.html
 The most important FindBugs errors are:
 At AppInfoCollector.java:[line 467]: Argument of type String can't be handled 
by format specifier %d in 
org.apache.oozie.tools.diag.AppInfoCollector.storeLastCoordinators(File, int, 
int)
 At AppInfoCollector.java:[line 451]: Argument of type String can't be handled 
by format specifier %d in 
org.apache.oozie.tools.diag.AppInfoCollector.storeLastWorkflows(File, int, int)
 At AppInfoCollector.java:[line 483]: Argument of type String can't be handled 
by format specifier %d in 
org.apache.oozie.tools.diag.AppInfoCollector.stroreLastBundles(File, int, int)
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2049
Tests rerun: 71
Tests failed at first run: 
org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC,org.apache.oozie.action.hadoop.TestJavaActionExecutor,
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/80/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 3706k  100 3706k0 0  5084k  0 --:--:-- --:--:-- --:--:-- 5084k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-2296
Archiving artifacts
[Fast Archiver] Compressed 1.80 MB of artifacts by 74.7% relative to #75
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool

2017-10-05 Thread Peter Cseh (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192868#comment-16192868
 ] 

Peter Cseh commented on OOZIE-2296:
---

I think the leftover comments are small enough to commit the latest version and 
fix minor things down the road as they come up.
I've retriggered the pre-commit hook for the latest patch as the test failures 
don't seem related.
+1 (pending Jenkins)

> Add an Oozie diagnostic bundle tool
> ---
>
> Key: OOZIE-2296
> URL: https://issues.apache.org/jira/browse/OOZIE-2296
> Project: Oozie
>  Issue Type: New Feature
>  Components: tools
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
> Attachments: OOZIE-2296-002.patch, OOZIE-2296-003.patch, 
> OOZIE-2296-004.patch, OOZIE-2296-005.patch, OOZIE-2296-006.patch, 
> OOZIE-2296-009.patch, OOZIE-2296-010.patch, OOZIE-2296-011.patch, 
> OOZIE-2296-012.patch
>
>
> To help with our support cases, I've built a tool that collects a bunch of 
> job and other information from Oozie that throws it all in a tarball.  The 
> idea is that the user can just click a button in Cloudera Manager, and it 
> will run this tool.  
> This tool could be useful for others as an easy way to get information out of 
> Oozie, so I thought I'd contribute it here.  It's built as a "tool" (so it 
> sits next to the sharelib and database tools), and simply uses the Oozie 
> client for getting pretty much everything, so it doesn't require anything 
> special.
> Here's the information that it can get:
> # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist 
> }}
> # Oozie Server's resolved loaded configuration (from admin endpoint)
> # Other admin commands output (queue dump, env vars, etc)
> # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp)
> # Details from last n jobs and/or specific list of jobs
> #- job.properties contents
> #- XML definition
> #- verbose status for each job and each action etc
> #- Oozie logs
> #- Unfortunately, we can't get the launcher jobs' logs from Hadoop
> # Metrics/Instrumentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2871) when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate via:[TOKEN, KERBEROS]”

2017-10-05 Thread Denes Bodo (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192814#comment-16192814
 ] 

Denes Bodo commented on OOZIE-2871:
---

[~yangfang] Were you able to solve this issue?

> when Enable Kerberos,  Oozie perform tasks throw “Client cannot authenticate 
> via:[TOKEN, KERBEROS]”
> ---
>
> Key: OOZIE-2871
> URL: https://issues.apache.org/jira/browse/OOZIE-2871
> Project: Oozie
>  Issue Type: Bug
>  Components: security
>Affects Versions: 4.2.0
> Environment: Oozie version :4.2.0
> Hadoop version:2.7.2
> Both Oozie and Hadoop are enabled kerberos.
>Reporter: yangfang
>Priority: Critical
>
> When  Oozie and Hadoop both enabled kerberos， I submitted a mapreduce   job 
> to oozie，then I got the error as below：
> 2017-04-27 13:37:12,677 WARN MapReduceActionExecutor: 523 - SERVER[zdh143] 
> USER[mr] GROUP[-] TOKEN[] APP[map-reduce-wf] 
> JOB[008-170427133546167-oozie-mr-W] 
> ACTION[008-170427133546167-oozie-mr-W@mr-node] Launcher exception: Failed 
> on local exception: java.io.IOException: 
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate 
> via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; 
> destination host is: "zdh143":9000; 
> java.io.IOException: Failed on local exception: java.io.IOException: 
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate 
> via:[TOKEN, KERBEROS]; Host Details : local host is: "zdh142/10.43.183.142"; 
> destination host is: "zdh143":9000; 
>   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:773)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1479)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1412)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
>   at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
>   at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>   at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source)
>   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
>   at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
>   at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)
>   at 
> org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:130)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)
>   at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1299)
>   at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1296)
>   at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
>   at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
>   at 
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
>   at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
>   at 
> org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:102)
>   at 
> org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:64)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
>   at 
> org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:38)
>   at

[jira] [Updated] (OOZIE-2406) Completely rewrite GraphGenerator code

2017-10-05 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2406:

Attachment: OOZIE-2406.011.patch

JUnit test reorganization, timeout for Graphviz rendering, addressing minor 
comments.

> Completely rewrite GraphGenerator code
> --
>
> Key: OOZIE-2406
> URL: https://issues.apache.org/jira/browse/OOZIE-2406
> Project: Oozie
>  Issue Type: New Feature
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: graphviz-10-action-nodes.png, 
> graphviz-25-action-nodes.png, graphviz-50-action-nodes.png, 
> graphviz-many-nodes.png, jung-many-nodes.png, OOZIE-2406.004.patch, 
> OOZIE-2406.004.patch, OOZIE-2406.005.patch, OOZIE-2406.006.patch, 
> OOZIE-2406.007.patch, OOZIE-2406.008.patch, OOZIE-2406.009.patch, 
> OOZIE-2406.010.patch, OOZIE-2406.011.patch, Screen Shot 2017-10-02 at 1.55.58 
> PM.png, Screen Shot 2017-10-02 at 1.56.33 PM.png
>
>
> The Web UI currently generates a graph of the workflow DAG as a png image 
> that it can show to the user ({{GraphGenerator}} class).  Unfortunately, 
> there are a number of downsides to the current implementation:
> # The image is generated server-side, which doesn't scale well and eats up 
> lots of memory
> #- To help combat this issue, we only generate graphs for workflows with less 
> than 25 nodes and we disabled the refresh button in the UI
> # It's slow
> # It uses a library ({{net.sf.jung}} from http://jung.sourceforge.net), which 
> hasn't been updated since 2010.  
> #- This library also has a dependency on a fork of Commons-Collections 
> ({{net.sourceforge.collections}} from 
> http://sourceforge.net/projects/collections/), which similarly hasn't been 
> updated since 2010.  This is a problem because we can't update 
> Commons-Collections when there are security concerns such as COLLECTIONS-580 
> (though Oozie is *not* susceptible to this attack).
> It would be good to do a complete rewrite, using a different library and 
> remove Jung and the Commons-Collections fork.  Whatever we choose, it should 
> an svg which will draw the image on the browser.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

2017-10-05 Thread András Piros


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62352/
---

(Updated Oct. 5, 2017, 11:02 a.m.)


Review request for oozie and Robert Kanter.


Changes
---

JUnit test reorganization, timeout for Graphviz rendering, addressing minor 
comments.


Repository: oozie-git


Description
---

OOZIE-2406 Completely rewrite GraphGenerator code


Diffs (updated)
-

  client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
74843dc9a3cbb8cef38e97f81e34731f191d9aab 
  core/pom.xml 6f9adb66af9344ac7d2212cdc31aa203ec06c286 
  core/src/main/java/org/apache/oozie/servlet/JsonRestServlet.java 
059d3cf6dc251b49940af29d82cbdd817043a176 
  core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 
10812c675ebc0cc4aefca9f4a85ef2fc26d143e1 
  core/src/main/java/org/apache/oozie/util/GraphGenerator.java 
6ded2c6dc15c9e8453ff800407ff0324be185f41 
  core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/graph/GraphRenderer.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/graph/GraphvizRenderer.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/graph/OutputFormat.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/graph/WorkflowActionNode.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/graph/WorkflowGraphHandler.java 
PRE-CREATION 
  core/src/main/resources/oozie-default.xml 
2389b993b7b1769521d6dd0c7386e99e7516210a 
  core/src/test/java/org/apache/oozie/servlet/TestV1JobServlet.java 
ee9ab556c9b6930c406ca5dcd54065cbbebb86c9 
  core/src/test/java/org/apache/oozie/util/TestGraphGenerator.java 
002e925b57cd830ea6d83a87cea4383165116b80 
  core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java 
PRE-CREATION 
  core/src/test/resources/graph-workflow-decision-fork-join.xml PRE-CREATION 
  core/src/test/resources/graph-workflow-simple.xml PRE-CREATION 
  core/src/test/resources/graphWF.xml 6a7b0427a9951835a7533a04b66258ded369d5bf 
  core/src/test/resources/graphWF_26_actions.xml 
a091be0f3559ede195ccc3339adee4478a8da8c0 
  core/src/test/resources/invalidGraphWF.xml  
  docs/src/site/twiki/WebServicesAPI.twiki 
ef3e60242512decd48beb3d8c9ac747b7d128eda 
  pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c 
  webapp/src/main/webapp/oozie-console.js 
72c8a198a4ffe60f74a9f700831f65efcb3066c4 


Diff: https://reviews.apache.org/r/62352/diff/7/

Changes: https://reviews.apache.org/r/62352/diff/6-7/


Testing
---

`TestGraphGenerator`, `TestV1JobServlet`


Thanks,

András Piros

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

2017-10-05 Thread András Piros



> On Oct. 2, 2017, 8:11 p.m., Peter Cseh wrote:
> > core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java
> > Lines 148 (patched)
> > 
> >
> > How is this test testing performance?
> 
> András Piros wrote:
> 5 graphs, 50+ nodes each... to render PNG images takes a bit of a time.
> 
> Peter Cseh wrote:
> So we're waiting to generate a bunch of images. We're asserting nothing. 
> Unless we hit the 2-hour timeout, this test would provides no information.

Test case removed.


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62352/#review186874
---


On Oct. 3, 2017, 9:05 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62352/
> ---
> 
> (Updated Oct. 3, 2017, 9:05 p.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2406 Completely rewrite GraphGenerator code
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 74843dc9a3cbb8cef38e97f81e34731f191d9aab 
>   core/pom.xml 6f9adb66af9344ac7d2212cdc31aa203ec06c286 
>   core/src/main/java/org/apache/oozie/servlet/JsonRestServlet.java 
> 059d3cf6dc251b49940af29d82cbdd817043a176 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 
> 10812c675ebc0cc4aefca9f4a85ef2fc26d143e1 
>   core/src/main/java/org/apache/oozie/util/GraphGenerator.java 
> 6ded2c6dc15c9e8453ff800407ff0324be185f41 
>   core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/GraphRenderer.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/GraphvizRenderer.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/OutputFormat.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/WorkflowActionNode.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/WorkflowGraphHandler.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobServlet.java 
> ee9ab556c9b6930c406ca5dcd54065cbbebb86c9 
>   core/src/test/java/org/apache/oozie/util/TestGraphGenerator.java 
> 002e925b57cd830ea6d83a87cea4383165116b80 
>   core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java 
> PRE-CREATION 
>   core/src/test/resources/graphWF.xml 
> 6a7b0427a9951835a7533a04b66258ded369d5bf 
>   core/src/test/resources/graphWF_26_actions.xml 
> a091be0f3559ede195ccc3339adee4478a8da8c0 
>   core/src/test/resources/graphWF_50_actions.xml PRE-CREATION 
>   core/src/test/resources/graphWF_decision_fork_join.xml PRE-CREATION 
>   docs/src/site/twiki/WebServicesAPI.twiki 
> ef3e60242512decd48beb3d8c9ac747b7d128eda 
>   examples/src/main/apps/java-main/workflow.xml 
> 98e01ca92187e1c567686f6c2b4f689bb2a5ef6a 
>   pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c 
>   webapp/src/main/webapp/oozie-console.js 
> 72c8a198a4ffe60f74a9f700831f65efcb3066c4 
> 
> 
> Diff: https://reviews.apache.org/r/62352/diff/6/
> 
> 
> Testing
> ---
> 
> `TestGraphGenerator`, `TestV1JobServlet`
> 
> 
> Thanks,
> 
> András Piros
> 
>

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

2017-10-05 Thread András Piros



> On Oct. 4, 2017, 10:07 a.m., Peter Cseh wrote:
> > core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java
> > Lines 54-56 (patched)
> > 
> >
> > Can this be null?

Test case removed.


> On Oct. 4, 2017, 10:07 a.m., Peter Cseh wrote:
> > core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java
> > Lines 64-65 (patched)
> > 
> >
> > Can this ever be null?

Test case removed.


> On Oct. 4, 2017, 10:07 a.m., Peter Cseh wrote:
> > core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java
> > Lines 108-109 (patched)
> > 
> >
> > Can you add an assert here that validates that it's a dot format and 
> > contains some information?

Yep, the same for SVGs.


> On Oct. 4, 2017, 10:07 a.m., Peter Cseh wrote:
> > core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java
> > Lines 127-140 (patched)
> > 
> >
> > Can you consolidate down the test cases? The testManyNodesX() can have 
> > a basic assertion for PNG, dot and svg.

Reorganized test cases, deleted and renamed a few of them in following classes:
- simple graph: whether graph generation and writing to output formats PNG, DOT 
or SVG is correct
- more sophisticated graph that doesn't have decision, fork, or join nodes: 
test that graph generation is correct
- even more sophisticated graph that has decision, fork, and join nodes: test 
that graph generation is correct


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62352/#review187077
---


On Oct. 3, 2017, 9:05 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62352/
> ---
> 
> (Updated Oct. 3, 2017, 9:05 p.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2406 Completely rewrite GraphGenerator code
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 74843dc9a3cbb8cef38e97f81e34731f191d9aab 
>   core/pom.xml 6f9adb66af9344ac7d2212cdc31aa203ec06c286 
>   core/src/main/java/org/apache/oozie/servlet/JsonRestServlet.java 
> 059d3cf6dc251b49940af29d82cbdd817043a176 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 
> 10812c675ebc0cc4aefca9f4a85ef2fc26d143e1 
>   core/src/main/java/org/apache/oozie/util/GraphGenerator.java 
> 6ded2c6dc15c9e8453ff800407ff0324be185f41 
>   core/src/main/java/org/apache/oozie/util/graph/GraphGenerator.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/GraphRenderer.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/GraphvizRenderer.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/OutputFormat.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/WorkflowActionNode.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/graph/WorkflowGraphHandler.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobServlet.java 
> ee9ab556c9b6930c406ca5dcd54065cbbebb86c9 
>   core/src/test/java/org/apache/oozie/util/TestGraphGenerator.java 
> 002e925b57cd830ea6d83a87cea4383165116b80 
>   core/src/test/java/org/apache/oozie/util/graph/TestGraphGenerator.java 
> PRE-CREATION 
>   core/src/test/resources/graphWF.xml 
> 6a7b0427a9951835a7533a04b66258ded369d5bf 
>   core/src/test/resources/graphWF_26_actions.xml 
> a091be0f3559ede195ccc3339adee4478a8da8c0 
>   core/src/test/resources/graphWF_50_actions.xml PRE-CREATION 
>   core/src/test/resources/graphWF_decision_fork_join.xml PRE-CREATION 
>   docs/src/site/twiki/WebServicesAPI.twiki 
> ef3e60242512decd48beb3d8c9ac747b7d128eda 
>   examples/src/main/apps/java-main/workflow.xml 
> 98e01ca92187e1c567686f6c2b4f689bb2a5ef6a 
>   pom.xml 0b94484da1c97618e9168cea0ebbfff7f70f723c 
>   webapp/src/main/webapp/oozie-console.js 
> 72c8a198a4ffe60f74a9f700831f65efcb3066c4 
> 
> 
> Diff: https://reviews.apache.org/r/62352/diff/6/
> 
> 
> Testing
> ---
> 
> `TestGraphGenerator`, `TestV1JobServlet`
> 
> 
> Thanks,
> 
> András Piros
> 
>

[jira] [Commented] (OOZIE-3077) SLA documentation fixes

2017-10-05 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192557#comment-16192557
 ] 

Andras Piros commented on OOZIE-3077:
-

Thanks for the patch [~satishsaley]! +1 (non-binding)

> SLA documentation fixes
> ---
>
> Key: OOZIE-3077
> URL: https://issues.apache.org/jira/browse/OOZIE-3077
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Attachments: OOZIE-3077-1.patch
>
>
> Following properties can be specified in job.xml -> Following properties can 
> be specified in job.xml, job.properties
> Default is 24 hours -> Default is 48 hours
> oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the 
> example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3005) Cover all workflow action types and workflow / action SLA with JAXB

2017-10-05 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3005:

Remaining Estimate: 8h
 Original Estimate: 8h

> Cover all workflow action types and workflow / action SLA with JAXB
> ---
>
> Key: OOZIE-3005
> URL: https://issues.apache.org/jira/browse/OOZIE-3005
> Project: Oozie
>  Issue Type: Sub-task
>  Components: client
>Reporter: Daniel Becker
>Assignee: Andras Piros
>   Original Estimate: 8h
>  Remaining Estimate: 8h
>
> All the Oozie workflow actions have to be covered, not only MapReduce ones, 
> considering following:
> * no XSD change should be performed
> * XJB mapping file has to be extended to incorporate all the workflow action 
> types, also the ones inside
> * Use the same namespace prefixes / namespaces in the XMLs as in the Oozie 
> examples.
> * * 
> {{//xs:complexType[@name='ACTION']/xs:sequence/xs:choice/xs:any[@namespace='##other']}}
> * XJB mapping file has to be extended to incorporate SLA related stuff inside 
> {{//xs:complexType[@name='ACTION']/xs:sequence/xs:any[@namespace='uri:oozie:sla:0.1
>  uri:oozie:sla:0.2']}}
> * autogenerate sources from different XSDs to different packages, resolve 
> namespace clashes that way. Note that it's not a goal to have all namespace 
> clashes resolve within XSDs
> * extend {{TestJAXBWorkflow}} to handle all those different action types, and 
> SLA, as well



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null

2017-10-05 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16192565#comment-16192565
 ] 

Andras Piros commented on OOZIE-3079:
-

Thanks for the fix [~satishsaley]! +1 (non-binding).

Can you please re-upload the patch so that a new Jenkins build is triggered and 
{{TESTS}} build target gets one more run?

> Filtering coordinators returns bundle id as null
> 
>
> Key: OOZIE-3079
> URL: https://issues.apache.org/jira/browse/OOZIE-3079
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-3079-1.patch
>
>
> If we filter coordinator jobs, the json reply shows (parent) bundle id as 
> null even though coordinator was started from a bundle.
> Example query - 
> http://localhost:11000/oozie/v1/jobs?filter=user=saley=coordinator



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (OOZIE-3017) API for workflows: other action types

2017-10-05 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3017:
---

Assignee: Andras Piros  (was: Daniel Becker)

> API for workflows: other action types
> -
>
> Key: OOZIE-3017
> URL: https://issues.apache.org/jira/browse/OOZIE-3017
> Project: Oozie
>  Issue Type: Sub-task
>  Components: client
>Reporter: Daniel Becker
>Assignee: Andras Piros
>   Original Estimate: 32h
>  Time Spent: 16h
>  Remaining Estimate: 16h
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3032) API for workflows: decision nodes autogeneration (difficult cases)

2017-10-05 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3032:

Remaining Estimate: 16h
 Original Estimate: 16h

> API for workflows: decision nodes autogeneration (difficult cases)
> --
>
> Key: OOZIE-3032
> URL: https://issues.apache.org/jira/browse/OOZIE-3032
> Project: Oozie
>  Issue Type: Sub-task
>  Components: client
>Reporter: Andras Piros
> Attachments: 
> 20170825093712-incoming-conditional-branches-from-different-decisions-workflow.png,
>  partial_dag.png
>
>   Original Estimate: 16h
>  Remaining Estimate: 16h
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] Subscription: Oozie Patch Available

2017-10-05 Thread jira

Issue Subscription
Filter: Oozie Patch Available (110 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3079  Filtering coordinators returns bundle id as null
https://issues.apache.org/jira/browse/OOZIE-3079
OOZIE-3078  PasswordMasker throws NPE with null arguments
https://issues.apache.org/jira/browse/OOZIE-3078
OOZIE-3077  SLA documentation fixes
https://issues.apache.org/jira/browse/OOZIE-3077
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3031  Coord job with only unresolved dependencies doesn't timeout
https://issues.apache.org/jira/browse/OOZIE-3031
OOZIE-3022  fix for warning has no file and won't be listed in dependency files 
details
https://issues.apache.org/jira/browse/OOZIE-3022
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-3001  core library has many instances of warnings with trailing spaces 
and lines longer than 132 chars
https://issues.apache.org/jira/browse/OOZIE-3001
OOZIE-2997  files contain trailing white spaces in client lib
https://issues.apache.org/jira/browse/OOZIE-2997
OOZIE-2996  add option for -UseGCOverheadLimit to maven opts as sometimes local 
testing fails
https://issues.apache.org/jira/browse/OOZIE-2996
OOZIE-2978  Remove code that handles Pig versions before 0.8 
https://issues.apache.org/jira/browse/OOZIE-2978
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2969  Drop support for Java 1.7
https://issues.apache.org/jira/browse/OOZIE-2969
OOZIE-2964  Add -Xdoclint:none to javadoc opts to avoid warnings
https://issues.apache.org/jira/browse/OOZIE-2964
OOZIE-2962  bump maven-javadoc-plugin to 2.10.4
https://issues.apache.org/jira/browse/OOZIE-2962
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2812

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

Re: Review Request 62459: OOZIE-2296: Add an Oozie diagnostic bundle tool

Re: Review Request 62459: OOZIE-2296: Add an Oozie diagnostic bundle tool

[jira] [Resolved] (OOZIE-1723) Setting 'use.new.api=true' gives compatibility error even with using new 'mapreduce.*' hadoop api

[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments

[jira] [Comment Edited] (OOZIE-3078) PasswordMasker throws NPE with null arguments

[jira] [Commented] (OOZIE-3078) PasswordMasker throws NPE with null arguments

[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null

Failed: OOZIE-3079 PreCommit Build #81

[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null

[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0

[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0

[jira] [Updated] (OOZIE-3079) Filtering coordinators returns bundle id as null

[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0

[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool

Failed: OOZIE-2296 PreCommit Build #80

[jira] [Commented] (OOZIE-2296) Add an Oozie diagnostic bundle tool

[jira] [Commented] (OOZIE-2871) when Enable Kerberos, Oozie perform tasks throw “Client cannot authenticate via:[TOKEN, KERBEROS]”

[jira] [Updated] (OOZIE-2406) Completely rewrite GraphGenerator code

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

Re: Review Request 62352: OOZIE-2406 Completely rewrite GraphGenerator code

[jira] [Commented] (OOZIE-3077) SLA documentation fixes

[jira] [Updated] (OOZIE-3005) Cover all workflow action types and workflow / action SLA with JAXB

[jira] [Commented] (OOZIE-3079) Filtering coordinators returns bundle id as null

[jira] [Assigned] (OOZIE-3017) API for workflows: other action types

[jira] [Updated] (OOZIE-3032) API for workflows: decision nodes autogeneration (difficult cases)

[jira] Subscription: Oozie Patch Available

28 matches

Site Navigation

Mail list logo

Footer information