date:20171130

Re: Review Request 63837: SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Peter Cseh via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63837/
---

(Updated Nov. 30, 2017, 1:18 p.m.)


Review request for oozie.


Bugs: OOZIE-3112
https://issues.apache.org/jira/browse/OOZIE-3112


Repository: oozie-git


Description
---

SparkConfigurationService injects the config values as --conf to SparkSubmit. 
This will overwrite propties provided in the --properies-file option which is 
not the expected behavior.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
2338ad28bf1a57b2fceab586372b237973d90f80 
  docs/src/site/twiki/DG_SparkActionExtension.twiki 
45b60b86bee4cfe2de2d374f03b602ddacdc5e65 
  sharelib/spark/pom.xml 65f641a927ea2fe3f357333dc0fcd518c2ad270f 
  
sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java
 052950fa36a0b40f8629ce91c1d135062a6571a9 
  
sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
 d97f1f06b5832b1f09fab30b08ecff31db3221f6 
  
sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkArgsExtractor.java
 574bf240304379d5d26af8a3e5c079d4ad707b7b 


Diff: https://reviews.apache.org/r/63837/diff/6/

Changes: https://reviews.apache.org/r/63837/diff/5-6/


Testing
---


Thanks,

Peter Cseh

[jira] [Updated] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-3112:
--
Attachment: OOZIE-3112.08.patch

Fixing RB comments and Findbugs things

> SparkConfigrationService overwrites properties provided via --properties-file 
> option in SparkAction
> ---
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch, 
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch, 
> OOZIE-3112.06.patch, OOZIE-3112.07.patch, OOZIE-3112.08.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (OOZIE-3135) Configure log4j2 in SqoopMain

2017-11-30 Thread Attila Sasvari (JIRA)

Attila Sasvari created OOZIE-3135:
-

 Summary: Configure log4j2 in SqoopMain
 Key: OOZIE-3135
 URL: https://issues.apache.org/jira/browse/OOZIE-3135
 Project: Oozie
  Issue Type: Bug
Affects Versions: 5.0.0b1
Reporter: Attila Sasvari


In Hadoop 3, MAPREDUCE-6983 switched to use slfj4 & log4j2 in 
{{org.apache.hadoop.mapreduce.Job}} (that prints out MR job id-s needed for 
Oozie). We need to setup log4j accordingly (it is also related to HADOOP-12956).
Without proper configuration in the Sqoop action, we won't be able to get 
external job id-s (SqoopActionExecutor unit tests and real action would be also 
affected).
   
[The API for Log4j 2 is not compatible with Log4j 
1.x|https://logging.apache.org/log4j/2.x/], but we will need to support both 
hadoop 2 and hadoop 3 profiles for a while. 

We could use reflection to determine the type of the logger object in 
{{org.apache.hadoop.mapreduce.Job}} and configure log4j settings based on it, 
but there might be a better way.

For example we could do something like this:
- add a new method for configuring log4j2:
{code}
private String setUpSqoopLog4J2(final String rootLogLevel) throws 
IOException {
System.out.println("Setting up log4j2");

final String logFile = getSqoopLogFile();
final File log4j2Xml = new File(SQOOP_LOG4J2_XML);
try (Writer writer = new FileWriter(log4j2Xml))
{
final String logj2SettingsXml = "\n" +
"\n" +
"\n" +
"\n" +
"\n" +
"\n" +
"  
\n" +
"\n" +
" \n" +
"\n" +
"\n" +
"\n" +
"\n" +
"\n" +
"\n" +
"\n" +
"";
writer.write(logj2SettingsXml);
}

System.out.printf("log4j2 configuration file created at %s%n", 
log4j2Xml.getAbsolutePath());

final   LoggerContext context = (LoggerContext) 
LogManager.getContext(false);
context.setConfigLocation(log4j2Xml.toURI()); // forces log4j2 
reconfiguration
return logFile;
}
{code}
and call it in the {{run()}} method if the mapreduce client is using slf4j for 
logging:
{code}
String logFile;
// MAPREDUCE-6983 switches to slfj4 & log4j2. Need to setup log4j 
accordingly
if 
(org.apache.hadoop.mapreduce.Job.class.getDeclaredField("LOG").getType().
isAssignableFrom(org.slf4j.Logger.class)) {
logFile = setUpSqoopLog4J2(rootLogLevel);
}
else {
logFile = setUpSqoopLog4J(rootLogLevel, logLevel);
}
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Attila Sasvari (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272668#comment-16272668
 ] 

Attila Sasvari commented on OOZIE-3112:
---

+1, pending jenkins

> SparkConfigrationService overwrites properties provided via --properties-file 
> option in SparkAction
> ---
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch, 
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch, 
> OOZIE-3112.06.patch, OOZIE-3112.07.patch, OOZIE-3112.08.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Attila Sasvari (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272672#comment-16272672
 ] 

Attila Sasvari commented on OOZIE-3112:
---

Can you fix the whitespace in SparkActionExecutor?

FROM
{code}
if(!sparkConfig.isEmpty()) {

{code}
TO
{code}
if (!sparkConfig.isEmpty()) {
{code}
( 
https://issues.apache.org/jira/secure/attachment/12900010/OOZIE-3112.08.patch#file-0
 )


> SparkConfigrationService overwrites properties provided via --properties-file 
> option in SparkAction
> ---
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch, 
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch, 
> OOZIE-3112.06.patch, OOZIE-3112.07.patch, OOZIE-3112.08.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63837: SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread András Piros via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63837/#review192297
---


Ship it!




Ship It!

- András Piros


On Nov. 30, 2017, 1:18 p.m., Peter Cseh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63837/
> ---
> 
> (Updated Nov. 30, 2017, 1:18 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-3112
> https://issues.apache.org/jira/browse/OOZIE-3112
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
> 2338ad28bf1a57b2fceab586372b237973d90f80 
>   docs/src/site/twiki/DG_SparkActionExtension.twiki 
> 45b60b86bee4cfe2de2d374f03b602ddacdc5e65 
>   sharelib/spark/pom.xml 65f641a927ea2fe3f357333dc0fcd518c2ad270f 
>   
> sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java
>  052950fa36a0b40f8629ce91c1d135062a6571a9 
>   
> sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
>  d97f1f06b5832b1f09fab30b08ecff31db3221f6 
>   
> sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkArgsExtractor.java
>  574bf240304379d5d26af8a3e5c079d4ad707b7b 
> 
> 
> Diff: https://reviews.apache.org/r/63837/diff/6/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272796#comment-16272796
 ] 

Hadoop QA commented on OOZIE-3112:
--

Testing JIRA OOZIE-3112

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be 
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2075
.Tests failed: 5
.Tests errors: 0

.The patch failed the following testcases:

.  
testYarnClientFilesAndArchivesArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
.  
testDriverClassPathArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
.  
testYarnClientExecutorAndDriverExtraClasspathsArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
.  
testLocalClientArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
.  
testQuotedDriverAndExecutorExtraJavaOptionsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)

.Tests failing with errors:
.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/267/

> SparkConfigrationService overwrites properties provided via --properties-file 
> option in SparkAction
> ---
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch, 
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch, 
> OOZIE-3112.06.patch, OOZIE-3112.07.patch, OOZIE-3112.08.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-3112 PreCommit Build #267

2017-11-30 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-3112
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/267/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.69 MB...]
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[WARN] There are [1] new bugs found in total that would be nice to have fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2245 bytes
[TRACE] Full summary file size is 1378 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3112

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch does adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
0 There are [1] new bugs found in total that would be nice to have fixed.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 0 There are [1] new bugs found in [core] that would be nice to have fixed.
 You can find the FindBugs diff here: core/findbugs-new.html
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2075
Tests failed: 5
Tests errors: 0

The patch failed the following testcases:

  
testYarnClientFilesAndArchivesArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
  
testDriverClassPathArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
  
testYarnClientExecutorAndDriverExtraClasspathsArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
  
testLocalClientArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
  
testQuotedDriverAndExecutorExtraJavaOptionsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)

Tests failing with errors:
  

+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/267/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100 3706k  100 3706k0 0  3515k  0  0:00:01  0:00:01 --:--:-- 6863k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3112
Archiving artifacts
[Fast Archiver] Compressed 1.87 MB of artifacts by 78.7% relative to #257
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
5 tests failed.
FAILED:  
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testYarnClientFilesAndArchivesArgsParsing

Error Message:
Spark args mismatch expected:<[--master, yarn, --deploy-mode, clie

Re: Review Request 63739: OOZIE-3113 Retry for ZK lock release

2017-11-30 Thread András Piros via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63739/#review192309
---

core/src/main/java/org/apache/oozie/service/ZKLocksService.java
Lines 59 (patched)

You need `CONF_PREFIX` here.

core/src/main/java/org/apache/oozie/service/ZKLocksService.java
Lines 209-211 (patched)

While I understand this `Exception` handling is about to mimic previous 
behavior, I'd go on and investigate whether we can `catch` separate `Exception` 
subclasses, and whether it's OK to swallow these.

core/src/main/java/org/apache/oozie/service/ZKLocksService.java
Lines 222 (patched)

I find having this variable as seconds would be more flexible for the user. 
Currently no means of waiting e.g. 5 seconds between ZooKeeper outages.

core/src/main/java/org/apache/oozie/service/ZKLocksService.java
Line 215 (original), 236 (patched)

I'd only retry when there is a ZooKeeper connectivity issue, not on other 
types of `Exception`s.

core/src/main/java/org/apache/oozie/service/ZKLocksService.java
Lines 239 (patched)

Yep, blocking a thread for 30-ish minutes doesn't seem like a good idea. 
E.g. ZooKeeper outage lasts for only a few seconds under heavy load - it can 
happen that Oozie JVM ends up blocking lots of `Thread`s in `TIMED_WAITING` 
state.

Can you think of some, more elaborate solution? Maybe having an 
`ExecutorService` just to try to unlock, and `Callable` instances that would 
try unlocking. In that case, the number of `Thread`s blocked in `TIMED_WAITING` 
state would be:
* configurable
* limited to a manageable extent

core/src/test/java/org/apache/oozie/service/TestZKLocksService.java
Lines 563 (patched)

What about `Services.get().get(ZKLocksService.class)`?

core/src/test/java/org/apache/oozie/service/TestZKLocksService.java
Lines 567-574 (patched)

Extract method `setupSilentLock()`.

core/src/test/java/org/apache/oozie/service/TestZKLocksService.java
Lines 579-598 (patched)

Extract nested class w/ a catchy name.

core/src/test/java/org/apache/oozie/service/TestZKLocksService.java
Lines 594 (patched)

`LOG.error(e)`.

- András Piros

On Nov. 27, 2017, 7:22 p.m., Satish Saley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63739/
> ---
> 
> (Updated Nov. 27, 2017, 7:22 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-3113
> https://issues.apache.org/jira/browse/OOZIE-3113
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-3113 Retry for ZK lock release
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/service/ZKLocksService.java 2c71c001f 
>   core/src/main/resources/oozie-default.xml 8285df0a7 
>   core/src/test/java/org/apache/oozie/service/TestZKLocksService.java 
> d04f04e80 
> 
> 
> Diff: https://reviews.apache.org/r/63739/diff/2/
> 
> 
> Testing
> ---
> 
> Tested locally
> 
> 
> Thanks,
> 
> Satish Saley
> 
>

Re: Review Request 63837: SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Peter Cseh via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63837/
---

(Updated Nov. 30, 2017, 9:19 p.m.)


Review request for oozie.


Bugs: OOZIE-3112
https://issues.apache.org/jira/browse/OOZIE-3112


Repository: oozie-git


Description
---

SparkConfigurationService injects the config values as --conf to SparkSubmit. 
This will overwrite propties provided in the --properies-file option which is 
not the expected behavior.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
2338ad28bf1a57b2fceab586372b237973d90f80 
  docs/src/site/twiki/DG_SparkActionExtension.twiki 
45b60b86bee4cfe2de2d374f03b602ddacdc5e65 
  sharelib/spark/pom.xml 65f641a927ea2fe3f357333dc0fcd518c2ad270f 
  
sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java
 052950fa36a0b40f8629ce91c1d135062a6571a9 
  
sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
 d97f1f06b5832b1f09fab30b08ecff31db3221f6 
  
sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkArgsExtractor.java
 574bf240304379d5d26af8a3e5c079d4ad707b7b 


Diff: https://reviews.apache.org/r/63837/diff/7/

Changes: https://reviews.apache.org/r/63837/diff/6-7/


Testing
---


Thanks,

Peter Cseh

[jira] [Updated] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-3112:
--
Attachment: OOZIE-3112.09.patch

I've made a silly mistake in the last patch so we've generated the 
properties-file all the time.
Fixed it.

> SparkConfigrationService overwrites properties provided via --properties-file 
> option in SparkAction
> ---
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch, 
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch, 
> OOZIE-3112.06.patch, OOZIE-3112.07.patch, OOZIE-3112.08.patch, 
> OOZIE-3112.09.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-2681) fix javadoc to compile on JDK8 again

2017-11-30 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2681:
--
Attachment: OOZIE-2681.14.patch

> fix javadoc to compile on JDK8 again
> 
>
> Key: OOZIE-2681
> URL: https://issues.apache.org/jira/browse/OOZIE-2681
> Project: Oozie
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 4.3.0
>Reporter: Robert Kanter
>Assignee: Peter Cseh
>Priority: Critical
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2681-00.patch, OOZIE-2681-01.patch, 
> OOZIE-2681-02.patch, OOZIE-2681-03.patch, OOZIE-2681-04.patch, 
> OOZIE-2681.06.patch, OOZIE-2681.07.patch, OOZIE-2681.08.patch, 
> OOZIE-2681.09.patch, OOZIE-2681.10.patch, OOZIE-2681.11.patch, 
> OOZIE-2681.12.patch, OOZIE-2681.13.patch, OOZIE-2681.14.patch
>
>
> We have a lot of Javadoc build failures, which causes Oozie to fail to build 
> on Java 8, which is more strict than Java 6 or 7 was.  OOZIE-2178 fixed all 
> of these that existed at that time, but we've since introduced a lot more.  
> OOZIE-2487 made a workaround to prevent this from causing the build to fail, 
> but we should fix these (again) and remove the OOZIE-2487 workaround.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Success: OOZIE-3112 PreCommit Build #268

2017-11-30 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-3112
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/268/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.65 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive2].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[WARN] There are [1] new bugs found in total that would be nice to have fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2245 bytes
[TRACE] Full summary file size is 1378 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3112

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch does adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
0 There are [1] new bugs found in total that would be nice to have fixed.
 0 There are [1] new bugs found in [core] that would be nice to have fixed.
 You can find the FindBugs diff here: core/findbugs-new.html
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [tools].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2075
Tests rerun: 55
Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,
+1 DISTRO
+1 distro tarball builds with the patch 


+1 Overall result, good!, no -1s


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/268/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 3706k  100 3706k0 0  3460k  0  0:00:01  0:00:01 --:--:-- 3460k
Adding comment to JIRA
Comment added.

test-patch exit code: 0

[description-setter] Description set: OOZIE-3112
Archiving artifacts
[Fast Archiver] Compressed 1.86 MB of artifacts by 58.6% relative to #257
Recording test results
Email was triggered for: Success
Sending email for trigger: Success



#

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

2017-11-30 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273648#comment-16273648
 ] 

Hadoop QA commented on OOZIE-3112:
--

Testing JIRA OOZIE-3112

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
. {color:orange}0{color} There are [1] new bugs found in [core] that would be 
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2075
.Tests rerun: 55
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/268/

> SparkConfigrationService overwrites properties provided via --properties-file 
> option in SparkAction
> ---
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch, 
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch, 
> OOZIE-3112.06.patch, OOZIE-3112.07.patch, OOZIE-3112.08.patch, 
> OOZIE-3112.09.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. 
> This will overwrite propties provided in the --properies-file option which is 
> not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-2681 PreCommit Build #269

2017-11-30 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-2681
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/269/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.64 MB...]
[DEBUG] There are no new bugs found in [server].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2113 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-2681

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
+1 the patch does adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2074
Tests failed: 6
Tests errors: 1

The patch failed the following testcases:

  testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation)
  
testSqoopActionWithRedundantArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  
testSqoopActionWithArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  testSqoopEval(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  
testSqoopActionWithRedundantPrefix(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
  testSqoopAction(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)

Tests failing with errors:
  testHive2Action(org.apache.oozie.action.hadoop.TestHive2ActionExecutor)

+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/269/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 3706k  100 3706k0 0  5097k  0 --:--:-- --:--:-- --:--:-- 5097k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-2681
Archiving artifacts
[Fast Archiver] Compressed 1.76 MB of artifacts by 53.3% relative to #268
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
7 tests failed.
FAILED:  org.apache.oozie.util.TestMetricsInstrumentation.testJMXInstrumentation

Error Message:
Could not find own virtual machine

Stack Trace:
junit.framework.AssertionFailedError: Could not find own virtual machine
at 
org.apache.oozie.

[jira] [Commented] (OOZIE-2681) fix javadoc to compile on JDK8 again

2017-11-30 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273668#comment-16273668
 ] 

Hadoop QA commented on OOZIE-2681:
--

Testing JIRA OOZIE-2681

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2074
.Tests failed: 6
.Tests errors: 1

.The patch failed the following testcases:

.  testJMXInstrumentation(org.apache.oozie.util.TestMetricsInstrumentation)
.  
testSqoopActionWithRedundantArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  
testSqoopActionWithArgsAndFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  testSqoopEval(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  
testSqoopActionWithRedundantPrefix(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.  testSqoopAction(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)

.Tests failing with errors:
.  testHive2Action(org.apache.oozie.action.hadoop.TestHive2ActionExecutor)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/269/

> fix javadoc to compile on JDK8 again
> 
>
> Key: OOZIE-2681
> URL: https://issues.apache.org/jira/browse/OOZIE-2681
> Project: Oozie
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 4.3.0
>Reporter: Robert Kanter
>Assignee: Peter Cseh
>Priority: Critical
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2681-00.patch, OOZIE-2681-01.patch, 
> OOZIE-2681-02.patch, OOZIE-2681-03.patch, OOZIE-2681-04.patch, 
> OOZIE-2681.06.patch, OOZIE-2681.07.patch, OOZIE-2681.08.patch, 
> OOZIE-2681.09.patch, OOZIE-2681.10.patch, OOZIE-2681.11.patch, 
> OOZIE-2681.12.patch, OOZIE-2681.13.patch, OOZIE-2681.14.patch
>
>
> We have a lot of Javadoc build failures, which causes Oozie to fail to build 
> on Java 8, which is more strict than Java 6 or 7 was.  OOZIE-2178 fixed all 
> of these that existed at that time, but we've since introduced a lot more.  
> OOZIE-2487 made a workaround to prevent this from causing the build to fail, 
> but we should fix these (again) and remove the OOZIE-2487 workaround.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] Subscription: Oozie Patch Available

2017-11-30 Thread jira

Issue Subscription
Filter: Oozie Patch Available (111 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3129  Fix test TestConfigurationService.testOozieConfig
https://issues-test.apache.org/jira/browse/OOZIE-3129
OOZIE-3127  Remove redundant check for user
https://issues-test.apache.org/jira/browse/OOZIE-3127
OOZIE-3126  Add option to allow list of users to access system config
https://issues-test.apache.org/jira/browse/OOZIE-3126
OOZIE-3121  bump all maven plugins to latest versions
https://issues-test.apache.org/jira/browse/OOZIE-3121
OOZIE-3118  fix for error: self-closing element not allowed
https://issues-test.apache.org/jira/browse/OOZIE-3118
OOZIE-3117  fix for warning: no @throws for
https://issues-test.apache.org/jira/browse/OOZIE-3117
OOZIE-3116  fix for warning: no description for @throws
https://issues-test.apache.org/jira/browse/OOZIE-3116
OOZIE-3115  fix for javadoc warning: empty  tag
https://issues-test.apache.org/jira/browse/OOZIE-3115
OOZIE-3113  Retry for ZK lock release
https://issues-test.apache.org/jira/browse/OOZIE-3113
OOZIE-3112  SparkConfigrationService overwrites properties provided via 
--properties-file option in SparkAction
https://issues-test.apache.org/jira/browse/OOZIE-3112
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues-test.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues-test.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues-test.apache.org/jira/browse/OOZIE-3091
OOZIE-3083  Make improved version Info backward compatible
https://issues-test.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues-test.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues-test.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues-test.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues-test.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues-test.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues-test.apache.org/jira/browse/OOZIE-2973
OOZIE-2969  Drop support for Java 1.7
https://issues-test.apache.org/jira/browse/OOZIE-2969
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues-test.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues-test.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues-test.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues-test.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues-test.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues-test.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues-test.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues-test.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues-test.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues-test.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues-test.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues-test.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues-test.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful

[jira] Subscription: Oozie Patch Available

2017-11-30 Thread jira

Issue Subscription
Filter: Oozie Patch Available (110 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3129  Fix test TestConfigurationService.testOozieConfig
https://issues.apache.org/jira/browse/OOZIE-3129
OOZIE-3127  Remove redundant check for user
https://issues.apache.org/jira/browse/OOZIE-3127
OOZIE-3121  bump all maven plugins to latest versions
https://issues.apache.org/jira/browse/OOZIE-3121
OOZIE-3118  fix for error: self-closing element not allowed
https://issues.apache.org/jira/browse/OOZIE-3118
OOZIE-3117  fix for warning: no @throws for
https://issues.apache.org/jira/browse/OOZIE-3117
OOZIE-3116  fix for warning: no description for @throws
https://issues.apache.org/jira/browse/OOZIE-3116
OOZIE-3115  fix for javadoc warning: empty  tag
https://issues.apache.org/jira/browse/OOZIE-3115
OOZIE-3113  Retry for ZK lock release
https://issues.apache.org/jira/browse/OOZIE-3113
OOZIE-3112  SparkConfigrationService overwrites properties provided via 
--properties-file option in SparkAction
https://issues.apache.org/jira/browse/OOZIE-3112
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3083  Make improved version Info backward compatible
https://issues.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues.apache.org/jira/browse/OOZIE-2973
OOZIE-2969  Drop support for Java 1.7
https://issues.apache.org/jira/browse/OOZIE-2969
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve s

Re: Review Request 63837: SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

[jira] [Updated] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

[jira] [Created] (OOZIE-3135) Configure log4j2 in SqoopMain

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

Re: Review Request 63837: SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

Failed: OOZIE-3112 PreCommit Build #267

Re: Review Request 63739: OOZIE-3113 Retry for ZK lock release

Re: Review Request 63837: SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

[jira] [Updated] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

[jira] [Updated] (OOZIE-2681) fix javadoc to compile on JDK8 again

Success: OOZIE-3112 PreCommit Build #268

[jira] [Commented] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

Failed: OOZIE-2681 PreCommit Build #269

[jira] [Commented] (OOZIE-2681) fix javadoc to compile on JDK8 again

[jira] Subscription: Oozie Patch Available

[jira] Subscription: Oozie Patch Available

18 matches

Site Navigation

Mail list logo

Footer information