[jira] Subscription: Oozie Patch Available

2017-11-21 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (115 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3132  Instrument SLACalculatorMemory
https://issues.apache.org/jira/browse/OOZIE-3132
OOZIE-3129  Fix test TestConfigurationService.testOozieConfig
https://issues.apache.org/jira/browse/OOZIE-3129
OOZIE-3127  Remove redundant check for user
https://issues.apache.org/jira/browse/OOZIE-3127
OOZIE-3126  Add option to allow list of users to access system config
https://issues.apache.org/jira/browse/OOZIE-3126
OOZIE-3121  bump all maven plugins to latest versions
https://issues.apache.org/jira/browse/OOZIE-3121
OOZIE-3118  fix for error: self-closing element not allowed
https://issues.apache.org/jira/browse/OOZIE-3118
OOZIE-3117  fix for warning: no @throws for
https://issues.apache.org/jira/browse/OOZIE-3117
OOZIE-3116  fix for warning: no description for @throws
https://issues.apache.org/jira/browse/OOZIE-3116
OOZIE-3115  fix for javadoc warning: empty  tag
https://issues.apache.org/jira/browse/OOZIE-3115
OOZIE-3114  fix for warning: no @return
https://issues.apache.org/jira/browse/OOZIE-3114
OOZIE-3113  Retry for ZK lock release
https://issues.apache.org/jira/browse/OOZIE-3113
OOZIE-3112  SparkConfigrationService overwrites properties provided via 
--properties-file option in SparkAction
https://issues.apache.org/jira/browse/OOZIE-3112
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3083  Make improved version Info backward compatible
https://issues.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues.apache.org/jira/browse/OOZIE-2973
OOZIE-2969  Drop support for Java 1.7
https://issues.apache.org/jira/browse/OOZIE-2969
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2945  Update SpotBugs to stable version after GA
https://issues.apache.org/jira/browse/OOZIE-2945
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2900  Retrieve tokens for oozie.launcher.mapreduce.job.hdfs-servers 
before submission
https://issues.apache.org/jira/browse/OOZIE-2900
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"

Re: Review Request 63875: OOZIE-2900 Retrieve tokens for oozie.launcher.mapreduce.job.hdfs-servers before submission

2017-11-21 Thread Peter Cseh via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63875/#review191662
---




core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1379-1386 (patched)


Can this part be pushed down to DistcpActionExecutor? It does not feel like 
other actions would have to work with these properties.
Also, please cosider adding some logging here, at least on debug level to 
make it easier to see what's happening.


- Peter Cseh


On Nov. 21, 2017, 3:49 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63875/
> ---
> 
> (Updated Nov. 21, 2017, 3:49 p.m.)
> 
> 
> Review request for oozie, Peter Bacsko and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Before Oozie on YARN, ``JobSubmitter`` from MapReduce (more precisely 
> ``TokenCache.obtainTokensForNamenodes``) took care of obtaining delegation 
> tokens for HDFS nodes specified by 
> ``oozie.launcher.mapreduce.job.hdfs-servers`` before submitting the Oozie 
> launcher job.
> 
> Oozie launcher is now a Yarn Application Master. It needs HDFS delegation 
> tokens to be able to copy files between secure clusters via the Oozie DistCp 
> action. 
> 
> Changes:
> - ``JavaActionExecutor`` was modified to handle Distcp related parameters 
> like (``oozie.launcher.mapreduce.job.hdfs-servers`` and 
> ``oozie.launcher.mapreduce.job.hdfs-servers.token-renewal.exclude``)
> - ``HDFSCredentials`` was changed to reuse 
> ``TokenCache.obtainTokensForNamenodes`` to obtain HDFS delegation tokens.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/HDFSCredentials.java 
> 92a7ebe9a7876b6400d80356d5c826e77575e2ab 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> a1df304914b73d406e986409a8053c2a48e1bd38 
> 
> 
> Diff: https://reviews.apache.org/r/63875/diff/3/
> 
> 
> Testing
> ---
> 
> Tested on a secure cluster that Oozie dist cp action can copy file from 
> another secure cluster where different Kerberos realm was used.
> 
> - workflow:
> ```
> 
> 
> 
> Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> 
> 
>   
> 
> oozie.launcher.mapreduce.job.dfs.namenode.kerberos.principal.pattern
> *
>   
> 
>   oozie.launcher.mapreduce.job.hdfs-servers
>   hdfs://oozie.test1.com:8020,hdfs://remote.test2.com:8020
> 
>  
> 
> 
> oozie.launcher.mapreduce.job.hdfs-servers.token-renewal.exclude
> remote.test2.com
> 
> 
>   hdfs://remote.test2.com:8020/tmp/1
>   hdfs://oozie.test1.com:8020/tmp/2
> 
> 
> 
> 
> 
> 
> ```
> 
> Prior to executing the workflow I had to setup cross realm trust between the 
> test secure clusters. It involved:
> - changing Kerberos configuration ``/etc/krb5.conf`` (adding realms and 
> setting additional properties like ``udp_preference_limit = 1``)
> - regenerating service credentials
> - changing HDFS settings (such as 
> ``dfs.namenode.kerberos.principal.pattern``) and setting hadoop auth to local 
> rule like ``RULE:[2:$1](.*)s/(.*)/$1/g``
> - additional configuration to enable trust between the test hadoop clusters
> 
> 
> Thanks,
> 
> Attila Sasvari
> 
>

[jira] [Commented] (OOZIE-2714) Detect conflicting resources during class loading

2017-11-21 Thread Peter Cseh (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261523#comment-16261523
 ] 

Peter Cseh commented on OOZIE-2714:
---

I think this would be a great tool to debug and fix classpath issues of various 
kinds. However, I don't think the default behavior should be to throw an error 
when a class appears twice on the classpath. Even if there is no performance 
hit, we should have this turned off (at least for the moment)
I think it would be better to:
# instead of hardcoding the  launcher classname, make it a property. 
(oozie.launcher.class.name or similar?)
# Instead of throwing an error at the first conflict, have an option to write 
out all of them.
# create some kind of documentation so users can find and use this

> Detect conflicting resources during class loading
> -
>
> Key: OOZIE-2714
> URL: https://issues.apache.org/jira/browse/OOZIE-2714
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
> Attachments: ClassLoaderTest.java, OOZIE-2714-POC01.patch, 
> OOZIE-2714-POC02.patch
>
>
> There are a bunch of issues in Oozie which are related to class loading. 
> The main problem is that the classpath is constructed in a way which is very 
> specific to Oozie:
> - Hadoop lib jars
> - Sharelib jars
> - User-defined jars
> Sometimes there is a conflict between sharelib and hadoop lib version. Also, 
> users can add their own jars which sometimes contain a different version of 
> popular libraries such as Guava, Apache commons, etc.
> We should be able to detect these conflicts and print exact error message so 
> that Oozie users can take appropriate actions to resolve the problem.
> A possible approach is the following:
> * start the execution of an action on a different thread
> * replace the thread's context classloader with a classloader which can 
> detect conflicts
> * when the JVM invokes the {{loadClass()}} method of the classloader, it  
> scans through the jars (which are available as {{URLClassPath}} objects). If 
> it finds the given resource in at least two jars, it can do different things 
> depending on the setup:
> ** throws an error immediately, mentioning the conflicting jars (this is 
> probably too strict - but still an option)
> ** loads the two resource into a byte array and compares them - it only 
> throws an error if there is difference
> ** compares the jars but only emits an error message if there is a conflict
> ** something else (user defined action?)
> Implementing such a classloader is not difficult and would greatly enhance 
> the supportability of Oozie. It could work in multiple modes depending on the 
> setup - perhaps being able to control it from a workflow config is desirable. 
> If there's any problem, we should be able to turn it off completely, too.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3132) Instrument SLACalculatorMemory

2017-11-21 Thread Andras Piros (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261322#comment-16261322
 ] 

Andras Piros commented on OOZIE-3132:
-

{{TestMetricsInstrumentation#testJMXInstrumentation()}} is known to be flaky. 
[~pbacsko] can you please review once more? Thanks!

> Instrument SLACalculatorMemory
> --
>
> Key: OOZIE-3132
> URL: https://issues.apache.org/jira/browse/OOZIE-3132
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3132.001.patch, OOZIE-3132.002.patch, 
> OOZIE-3132.002.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} 
> instances that have to be followed up on creating {{SLASummaryBean}} 
> instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like 
> {{1}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and 
> {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} 
> showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. 
> Cannot add or update new SLA Registration entry for job}} showing the add / 
> update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from 
> {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is 
> its the actual size - that is, whether the next add or update command will 
> succeed
> We need an {{Instrumentation.Counter}} instance that gets incremented when 
> there is an {{SLACalculatorMemory#slaMap#put()}} with a new entry added, and 
> gets decremented when there happens a {{SLACalculatorMemory#slaMap#remove()}} 
> with an existing entry removed. This counter will be automatically present 
> within REST interface, and Oozie client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2900) Retrieve tokens for oozie.launcher.mapreduce.job.hdfs-servers before submission

2017-11-21 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261188#comment-16261188
 ] 

Hadoop QA commented on OOZIE-2900:
--

Testing JIRA OOZIE-2900

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [2] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:orange}0{color} There are [2] new bugs found in [core] that would be 
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2069
.Tests failed: 6
.Tests errors: 0

.The patch failed the following testcases:

.  testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation)
.  
testSqoopActionWithRedundantArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  
testSqoopActionWithArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  testSqoopEval(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  
testSqoopActionWithRedundantPrefix(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  testSqoopAction(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)

.Tests failing with errors:
.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/250/

> Retrieve tokens for oozie.launcher.mapreduce.job.hdfs-servers before 
> submission
> ---
>
> Key: OOZIE-2900
> URL: https://issues.apache.org/jira/browse/OOZIE-2900
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Attila Sasvari
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2900-000.patch, OOZIE-2900-001.patch, 
> OOZIE-2900-02.patch, OOZIE-2900-03.patch
>
>
> We have to get tokens for oozie.launcher.mapreduce.job.hdfs-servers. Cannot 
> do cross cluster distcp without that.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-2900 PreCommit Build #250

2017-11-21 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-2900
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/250/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 1.64 MB...]
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2069
Tests failed: 6
Tests errors: 0

The patch failed the following testcases:

  testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation)
  
testSqoopActionWithRedundantArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  
testSqoopActionWithArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  testSqoopEval(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  
testSqoopActionWithRedundantPrefix(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  testSqoopAction(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)

Tests failing with errors:
  

+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/250/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  2 3706k2 797550 0   114k  0  0:00:32 --:--:--  0:00:32  
114k100 3706k  100 3706k0 0  4399k  0 --:--:-- --:--:-- --:--:-- 
21.7M
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-2900
Archiving artifacts
[Fast Archiver] Compressed 1.82 MB of artifacts by 43.0% relative to #242
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
6 tests failed.
FAILED:  
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopActionWithRedundantArgsAndFreeFormQuery

Error Message:
expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>

Stack Trace:
junit.framework.ComparisonFailure: expected:<[SUCCEED]ED> but 
was:<[FAILED/KILL]ED>
at 
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.runSqoopActionFreeFormQuery(TestSqoopActionExecutor.java:326)
at 
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopActionWithRedundantArgsAndFreeFormQuery(TestSqoopActionExecutor.java:303)


FAILED:  
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopActionWithArgsAndFreeFormQuery

Error Message:
expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>

Stack Trace:
junit.framework.ComparisonFailure: expected:<[SUCCEED]ED> but 
was:<[FAILED/KILL]ED>
at 
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.runSqoopActionFreeFormQuery(TestSqoopActionExecutor.java:326)
at 
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopActionWithArgsAndFreeFormQuery(TestSqoopActionExecutor.java:310)


FAILED:  org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopEval

Error Message:
expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>

Stack Trace:
junit.framework.ComparisonFailure: expected:<[SUCCEED]ED> but 
was:<[FAILED/KILL]ED>
at 
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopEval(TestSqoopActionExecutor.java:280)


FAILED:  
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.testSqoopActionWithRedundantPrefix

Error Message:
expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>

Stack Trace:
junit.framework.ComparisonFailure: expected:<[SUCCEED]ED> but 
was:<[FAILED/KILL]ED>
at 
org.apache.oozie.action.hadoop.TestSqoopActionExecutor.runSqoopAction(TestSqoopActionExecutor.java:245)
at

[jira] [Commented] (OOZIE-3132) Instrument SLACalculatorMemory

2017-11-21 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261027#comment-16261027
 ] 

Hadoop QA commented on OOZIE-3132:
--

Testing JIRA OOZIE-3132

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2071
.Tests failed: 1
.Tests errors: 0

.The patch failed the following testcases:

.  testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation)

.Tests failing with errors:
.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/249/

> Instrument SLACalculatorMemory
> --
>
> Key: OOZIE-3132
> URL: https://issues.apache.org/jira/browse/OOZIE-3132
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3132.001.patch, OOZIE-3132.002.patch, 
> OOZIE-3132.002.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} 
> instances that have to be followed up on creating {{SLASummaryBean}} 
> instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like 
> {{1}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and 
> {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} 
> showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. 
> Cannot add or update new SLA Registration entry for job}} showing the add / 
> update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from 
> {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is 
> its the actual size - that is, whether the next add or update command will 
> succeed
> We need an {{Instrumentation.Counter}} instance that gets incremented when 
> there is an {{SLACalculatorMemory#slaMap#put()}} with a new entry added, and 
> gets decremented when there happens a

Failed: OOZIE-3132 PreCommit Build #249

2017-11-21 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3132
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/249/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 1.64 MB...]
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2071
Tests failed: 1
Tests errors: 0

The patch failed the following testcases:

  testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation)

Tests failing with errors:
  

+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/249/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 3706k  100 3706k0 0  3538k  0  0:00:01  0:00:01 --:--:-- 3538k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3132
Archiving artifacts
[Fast Archiver] Compressed 1.82 MB of artifacts by 32.5% relative to #242
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  org.apache.oozie.util.TestMetricsInstrumentation.testJMXInstrumentation

Error Message:
Could not find own virtual machine

Stack Trace:
junit.framework.AssertionFailedError: Could not find own virtual machine
at 
org.apache.oozie.util.TestMetricsInstrumentation.testJMXInstrumentation(TestMetricsInstrumentation.java:243)

Failed: OOZIE-3132 PreCommit Build #248

2017-11-21 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3132
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/248/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 1.65 MB...]
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 77 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 0
Tests rerun: 0
Tests failed at first run: 
org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.command.coord.TestCoordSubmitXCommand,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.util.TestMetricsInstrumentation,org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand,org.apache.oozie.service.TestConfigurationService,
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/248/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 3706k0 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100 3706k  100 3706k0 0  3112k  0  0:00:01  0:00:01 --:--:-- 18.0M
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3132
Archiving artifacts
[Fast Archiver] Compressed 1.82 MB of artifacts by 53.3% relative to #242
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
8 tests failed.
FAILED:  
org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand.testCoordActionPullPushDependencyMissing

Error Message:
E0803: IO error, E0603: SQL error in operation, The transaction has been rolled 
back.  See the nested exceptions for details on the errors that occurred.

Stack Trace:
org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error in 
operation, The transaction has been rolled back.  See the nested exceptions for 
details on the errors that occurred.
at 
org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand.testCoordActionPullPushDependencyMissing(TestCoordActionMissingDependenciesXCommand.java:202)
Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in 
operation, The transaction has been rolled back.  See the nested exceptions for 
details on the errors that occurred.
at 
org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand.testCoordActionPullPushDependencyMissing(TestCoordActionMissingDependenciesXCommand.java:202)
Caused by: org.apache.oozie.executor.jpa.JPAExecutorException: E0603: SQL error 
in operation, The transaction has been rolled back.  See the nested exceptions 
for details on the errors that occurred.
at 
org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand.testCoordActionPullPushDependencyMissing(TestCoordActionMissingDependenciesXCommand.java:202)


FAILED:  
org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand.testCoordActionPushDependencyMissing

Error Message:
E0803: IO error, E0603: SQL error in

[jira] [Commented] (OOZIE-3132) Instrument SLACalculatorMemory

2017-11-21 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260958#comment-16260958
 ] 

Hadoop QA commented on OOZIE-3132:
--

Testing JIRA OOZIE-3132

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 1 line(s) with tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 4 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 0
.Tests rerun: 0
.Tests failed at first run: 
org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.command.coord.TestCoordSubmitXCommand,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.util.TestMetricsInstrumentation,org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand,org.apache.oozie.service.TestConfigurationService,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/248/

> Instrument SLACalculatorMemory
> --
>
> Key: OOZIE-3132
> URL: https://issues.apache.org/jira/browse/OOZIE-3132
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3132.001.patch, OOZIE-3132.002.patch, 
> OOZIE-3132.002.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} 
> instances that have to be followed up on creating {{SLASummaryBean}} 
> instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like 
> {{1}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and 
> {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} 
> showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. 
> Cannot add or update new SLA Registration entry for job}} showing the add / 
> update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from 
> {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is 
> its the actual size - that is, whether the next add or update command

[jira] [Updated] (OOZIE-2900) Retrieve tokens for oozie.launcher.mapreduce.job.hdfs-servers before submission

2017-11-21 Thread Attila Sasvari (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2900:
--
Attachment: OOZIE-2900-03.patch

disregard 02 patch

> Retrieve tokens for oozie.launcher.mapreduce.job.hdfs-servers before 
> submission
> ---
>
> Key: OOZIE-2900
> URL: https://issues.apache.org/jira/browse/OOZIE-2900
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Attila Sasvari
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2900-000.patch, OOZIE-2900-001.patch, 
> OOZIE-2900-02.patch, OOZIE-2900-03.patch
>
>
> We have to get tokens for oozie.launcher.mapreduce.job.hdfs-servers. Cannot 
> do cross cluster distcp without that.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3132) Instrument SLACalculatorMemory

2017-11-21 Thread Andras Piros (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3132:

Attachment: OOZIE-3132.002.patch

> Instrument SLACalculatorMemory
> --
>
> Key: OOZIE-3132
> URL: https://issues.apache.org/jira/browse/OOZIE-3132
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3132.001.patch, OOZIE-3132.002.patch, 
> OOZIE-3132.002.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} 
> instances that have to be followed up on creating {{SLASummaryBean}} 
> instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like 
> {{1}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and 
> {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} 
> showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. 
> Cannot add or update new SLA Registration entry for job}} showing the add / 
> update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from 
> {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is 
> its the actual size - that is, whether the next add or update command will 
> succeed
> We need an {{Instrumentation.Counter}} instance that gets incremented when 
> there is an {{SLACalculatorMemory#slaMap#put()}} with a new entry added, and 
> gets decremented when there happens a {{SLACalculatorMemory#slaMap#remove()}} 
> with an existing entry removed. This counter will be automatically present 
> within REST interface, and Oozie client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3132) Instrument SLACalculatorMemory

2017-11-21 Thread Andras Piros (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3132:

Attachment: OOZIE-3132.002.patch

> Instrument SLACalculatorMemory
> --
>
> Key: OOZIE-3132
> URL: https://issues.apache.org/jira/browse/OOZIE-3132
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3132.001.patch, OOZIE-3132.002.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} 
> instances that have to be followed up on creating {{SLASummaryBean}} 
> instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like 
> {{1}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and 
> {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} 
> showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. 
> Cannot add or update new SLA Registration entry for job}} showing the add / 
> update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from 
> {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is 
> its the actual size - that is, whether the next add or update command will 
> succeed
> We need an {{Instrumentation.Counter}} instance that gets incremented when 
> there is an {{SLACalculatorMemory#slaMap#put()}} with a new entry added, and 
> gets decremented when there happens a {{SLACalculatorMemory#slaMap#remove()}} 
> with an existing entry removed. This counter will be automatically present 
> within REST interface, and Oozie client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3132) Instrument SLACalculatorMemory

2017-11-21 Thread Andras Piros (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260741#comment-16260741
 ] 

Andras Piros commented on OOZIE-3132:
-

[~pbacsko] thanks for the review! Uploading new patch.

> Instrument SLACalculatorMemory
> --
>
> Key: OOZIE-3132
> URL: https://issues.apache.org/jira/browse/OOZIE-3132
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3132.001.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} 
> instances that have to be followed up on creating {{SLASummaryBean}} 
> instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like 
> {{1}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and 
> {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} 
> showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. 
> Cannot add or update new SLA Registration entry for job}} showing the add / 
> update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from 
> {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is 
> its the actual size - that is, whether the next add or update command will 
> succeed
> We need an {{Instrumentation.Counter}} instance that gets incremented when 
> there is an {{SLACalculatorMemory#slaMap#put()}} with a new entry added, and 
> gets decremented when there happens a {{SLACalculatorMemory#slaMap#remove()}} 
> with an existing entry removed. This counter will be automatically present 
> within REST interface, and Oozie client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)