[jira] Subscription: Oozie Patch Available

2019-02-15 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (92 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3418  Upgrade to Guava 27
https://issues.apache.org/jira/browse/OOZIE-3418
OOZIE-3409  Oozie Server : Memory leak in EL evaluation
https://issues.apache.org/jira/browse/OOZIE-3409
OOZIE-3404  The env variable of SPARK_HOME needs to be set when running pySpark
https://issues.apache.org/jira/browse/OOZIE-3404
OOZIE-3395  Findbugs is no longer maintained
https://issues.apache.org/jira/browse/OOZIE-3395
OOZIE-3375  Can't use empty  in coordinator
https://issues.apache.org/jira/browse/OOZIE-3375
OOZIE-3367  Using && in EL expressions in oozie bundle.xml files generates 
parse errors
https://issues.apache.org/jira/browse/OOZIE-3367
OOZIE-3366  Update workflow status and subworkflow status on suspend command
https://issues.apache.org/jira/browse/OOZIE-3366
OOZIE-3364  Rerunning Oozie bundle jobs starts the coordinators in 
indeterminate order
https://issues.apache.org/jira/browse/OOZIE-3364
OOZIE-3362  When killed, SSH action should kill the spawned processes on target 
host
https://issues.apache.org/jira/browse/OOZIE-3362
OOZIE-3326  Sqoop Action should support tez delegation tokens for hive-import
https://issues.apache.org/jira/browse/OOZIE-3326
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3137  Add support for log4j2 in HiveMain
https://issues.apache.org/jira/browse/OOZIE-3137
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2784  Include WEEK as a parameter in the Coordinator Expression Language 
Evaulator
https://issues.apache.org/jira/browse/OOZIE-2784
OOZIE-2779  Mask Hive2 action Beeline JDBC password
https://issues.apache.org/jira/browse/OOZIE-2779
OOZIE-2736  Reduce the number of threads during test execution
https://issues.apache.org/jira/browse/OOZIE-2736
OOZIE-2714  Detect conflicting resources during class loading

[jira] [Assigned] (OOZIE-3413) Cannot start Oozie Web UI after deleting /tmp/embedded-jetty-jsp

2019-02-15 Thread duan xiong (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

duan xiong reassigned OOZIE-3413:
-

Assignee: duan xiong

> Cannot start Oozie Web UI after deleting /tmp/embedded-jetty-jsp
> 
>
> Key: OOZIE-3413
> URL: https://issues.apache.org/jira/browse/OOZIE-3413
> Project: Oozie
>  Issue Type: Bug
>Reporter: Andras Salamon
>Assignee: duan xiong
>Priority: Major
>
> After the following steps:
>  # Restart oozie server using {{oozied.sh}}
>  # {{rm -rf /tmp/embedded-jetty-jsp}}
>  # Open the Oozie Web UI in a browser: {{[http://localhost:11000/oozie/]}}
> Oozie Web UI does not start, it gives the following error message:
> {noformat}
> {
>   "errorMessage": "org.apache.jasper.JasperException: 
> java.lang.ClassNotFoundException: org.apache.jsp.index_jsp",
>   "httpStatusCode": 500
> }{noformat}
> jetty.log contains the following error message:
> {noformat}
> 2018-12-28 10:16:19,630  WARN ServletHandler:624 - 
> org.apache.jasper.JasperException: java.lang.ClassNotFoundException: 
> org.apache.jsp.index_jsp
> at 
> org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:176)
> at 
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:375)
> at 
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:396)
> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340)
> at 
> org.eclipse.jetty.jsp.JettyJspServlet.service(JettyJspServlet.java:112)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:572)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:542)
> at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at 
> org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:566)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:199)
> at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
> at 
> org.eclipse.jetty.servlet.DefaultServlet.sendWelcome(DefaultServlet.java:599)
> at 
> org.eclipse.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:484)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at 
> org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
>

Failed: OOZIE-3249 PreCommit Build #1001

2019-02-15 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3249
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1001/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.89 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [fluent-job/fluent-job-api].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [webapp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2553 bytes
[TRACE] Full summary file size is 1471 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3249

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
-1 RAT
-1 the patch seems to introduce 1 new RAT warning(s)
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/git].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [core].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 3135
Tests failed at first run:
TestPurgeXCommand#testPurgeableBundleUnpurgeableCoordinatorUnpurgebleWorkflowPurgeableSubWorkflow
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/1001/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13158414/comment/16769501","id":"16769501","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small=hadoopqa=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall=hadoopqa=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium=hadoopqa=10393"},"displayName":"Hadoop

[jira] [Commented] (OOZIE-3249) [tools] Instrumentation log parser

2019-02-15 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769501#comment-16769501
 ] 

Hadoop QA commented on OOZIE-3249:
--


Testing JIRA OOZIE-3249

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 1 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3135
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeableBundleUnpurgeableCoordinatorUnpurgebleWorkflowPurgeableSubWorkflow
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/1001/



> [tools] Instrumentation log parser
> --
>
> Key: OOZIE-3249
> URL: https://issues.apache.org/jira/browse/OOZIE-3249
> Project: Oozie
>  Issue Type: Improvement
>  Components: tools
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3249.001.patch, 
> oozie-instrumentation-localhost.log.2018-05-09, 
> oozie-instrumentation-localhost.log.2018-05-09.out
>
>
> Oozie instrumentation logs contain a lot of information, but are difficult to 
> parse, because per instrumentation log entry there is always one header line 
> in plain text format (containing timestamp), and multiple other lines in JSON 
> format (not containing timestamp). Those lines of course belong together.
> {noformat}
> 2018-05-02 02:48:13,426  INFO oozieinstrumentation:520 - USER[-] GROUP[-] 
> TOKEN[-] APP[-] JOB[-] ACTION[-] 
> {
> ...
>   "counters" : {
> ...
> "callablequeue.executed" : {
>   "count" : 5954144
> },
> ...
> "callablequeue.queued" : {
>   "count" : 10596129
> },
> ...
>   },
> ...
> }
> {noformat}
> There should be a simple script in {{tools/bin}} that takes as parameters:
> * input file name ({{-i}}), e.g. {{-i

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769405#comment-16769405
 ] 

Andras Salamon commented on OOZIE-3326:
---

[~dionusos] I left a few comments at the review board.

> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OOZIE-3326_004.patch, 
> OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-3279) [fluent-job] API enhancements

2019-02-15 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3279:
---

Assignee: (was: Andras Piros)

> [fluent-job] API enhancements
> -
>
> Key: OOZIE-3279
> URL: https://issues.apache.org/jira/browse/OOZIE-3279
> Project: Oozie
>  Issue Type: New Feature
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Priority: Major
>
> All the features not part of the [Minimum Viable Fluent Job API|OOZIE-2339].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-2729) Use MiniMRYARNCluster in tests

2019-02-15 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-2729:
---

Assignee: (was: Andras Piros)

> Use MiniMRYARNCluster in tests
> --
>
> Key: OOZIE-2729
> URL: https://issues.apache.org/jira/browse/OOZIE-2729
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 4.3.0
>Reporter: Peter Cseh
>Priority: Major
> Attachments: OOZIE-2729.002.patch
>
>
> {{MiniMRCluster}} is deprecated, we should replace it using 
> {{MiniYARNCluster}}.
> Also {{MiniDFSCluster}} should be created via builder.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-2680) Increase HttpClient to 4.4

2019-02-15 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-2680:
---

Assignee: (was: Andras Piros)

> Increase HttpClient to 4.4
> --
>
> Key: OOZIE-2680
> URL: https://issues.apache.org/jira/browse/OOZIE-2680
> Project: Oozie
>  Issue Type: Bug
>Reporter: Robert Kanter
>Priority: Major
>
> Once Hadoop 2.8.0, 2.7.4, or 3.0.0-alpha1 becomes the minimum supported 
> Hadoop version, we should upgrade HttpClient to 4.4 (or a newer one if 
> available).  Previous versions are not compatible due to HADOOP-11859, which 
> was added in Hadoop 2.8.0.  See OOZIE-2679.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3249) [tools] Instrumentation log parser

2019-02-15 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769387#comment-16769387
 ] 

Andras Piros commented on OOZIE-3249:
-

[~kmarton] [~asalamon74] can you please review?

> [tools] Instrumentation log parser
> --
>
> Key: OOZIE-3249
> URL: https://issues.apache.org/jira/browse/OOZIE-3249
> Project: Oozie
>  Issue Type: Improvement
>  Components: tools
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3249.001.patch, 
> oozie-instrumentation-localhost.log.2018-05-09, 
> oozie-instrumentation-localhost.log.2018-05-09.out
>
>
> Oozie instrumentation logs contain a lot of information, but are difficult to 
> parse, because per instrumentation log entry there is always one header line 
> in plain text format (containing timestamp), and multiple other lines in JSON 
> format (not containing timestamp). Those lines of course belong together.
> {noformat}
> 2018-05-02 02:48:13,426  INFO oozieinstrumentation:520 - USER[-] GROUP[-] 
> TOKEN[-] APP[-] JOB[-] ACTION[-] 
> {
> ...
>   "counters" : {
> ...
> "callablequeue.executed" : {
>   "count" : 5954144
> },
> ...
> "callablequeue.queued" : {
>   "count" : 10596129
> },
> ...
>   },
> ...
> }
> {noformat}
> There should be a simple script in {{tools/bin}} that takes as parameters:
> * input file name ({{-i}}), e.g. {{-i /path/to/oozie-instrumentation.log}}
> * output file name ({{-o}}), e.g. {{-o 
> /path/to/oozie-instrumentation.log.out}}
> * parameters to extract ({{-p}}) in the format of 
> {{path/to/json/value1,path/to/json/value2}}, in this case {{-p 
> counters/callablequeue.executed/count,counters/callablequeue.queued/count}}
> The output file should contain in CSV format:
> * a header line containing column names for
> * one line per parsed input header / JSON lines, containing:
> ** first cell is the minutes part of the timestamp
> ** consecutive cells are parsed JSON values given each parameter to extract



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-2983) Stream the Launcher AM Logs

2019-02-15 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-2983:
---

Assignee: (was: Andras Piros)

> Stream the Launcher AM Logs
> ---
>
> Key: OOZIE-2983
> URL: https://issues.apache.org/jira/browse/OOZIE-2983
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Priority: Major
> Attachments: OOZIE-2983.001.patch
>
>
> Because OYA will get rid of the MR-based Launcher Job and have a custom AM 
> instead, the Launcher Job info and logs will no longer show up in the JHS. It 
> will show up in the ATSv2, however, the ATSv2 isn't quite ready yet and that 
> doesn't help users on older versions of Hadoop.  Launcher Jobs will stil show 
> up in the RM, but the RM doesn't remember things forever (and can be quite 
> short depending on cluster busyness and configs).
> We should have Oozie serve those logs itself (which would also be a nice 
> thing to have anyway, as the launcher logs are typically the most useful). 
> The code for reading those files is pretty easy to call. The yarn CLI, JHS, 
> and this [Aggregated Log Files 
> Exploder|https://github.com/rkanter/ALFExploder] tool I wrote all use it; the 
> latter proves that you can do it outside of Hadoop. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-2554) Spark action tests on yarn-cluster, yarn-client mode

2019-02-15 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-2554:
---

Assignee: (was: Andras Piros)

> Spark action tests on yarn-cluster, yarn-client mode
> 
>
> Key: OOZIE-2554
> URL: https://issues.apache.org/jira/browse/OOZIE-2554
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Satish Subhashrao Saley
>Priority: Major
> Attachments: OOZIE-2554.poc.patch
>
>
> Currently, we have unit test cases for Spark Action in local mode. To verify 
> other functionality, we should have tests in yarn-cluster and yarn-client 
> mode.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769263#comment-16769263
 ] 

Hadoop QA commented on OOZIE-3326:
--

PreCommit-OOZIE-Build started


> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OOZIE-3326_004.patch, 
> OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Denes Bodo (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denes Bodo updated OOZIE-3326:
--
Attachment: OOZIE-3326_004.patch

> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OOZIE-3326_004.patch, 
> OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3429) PreCommit FindBugs checking does not work

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769245#comment-16769245
 ] 

Andras Salamon commented on OOZIE-3429:
---

It was also working for this build: 
https://builds.apache.org/job/PreCommit-OOZIE-Build/999/artifact/test-patch/reports/TEST-SUMMARY.txt

> PreCommit FindBugs checking does not work
> -
>
> Key: OOZIE-3429
> URL: https://issues.apache.org/jira/browse/OOZIE-3429
> Project: Oozie
>  Issue Type: Bug
>Reporter: Andras Salamon
>Assignee: Julia Kinga Marton
>Priority: Major
>
> PreCommit FindBugs checking stopped reporting errors one or two months ago. 
> The first patch of OOZIE-3427 was 
> [reported|https://builds.apache.org/job/PreCommit-OOZIE-Build/992/consoleFull]
>  to be good:
> {noformat}+1 There are no new bugs found in total.
>  +1 There are no new bugs found in [client].
>  +1 There are no new bugs found in [server].
>  +1 There are no new bugs found in [fluent-job/fluent-job-api].
>  +1 There are no new bugs found in [webapp].
>  +1 There are no new bugs found in [examples].
>  +1 There are no new bugs found in [core].
>  +1 There are no new bugs found in [docs].
>  +1 There are no new bugs found in [tools].
>  +1 There are no new bugs found in [sharelib/distcp].
>  +1 There are no new bugs found in [sharelib/oozie].
>  +1 There are no new bugs found in [sharelib/hive].
>  +1 There are no new bugs found in [sharelib/spark].
>  +1 There are no new bugs found in [sharelib/pig].
>  +1 There are no new bugs found in [sharelib/hcatalog].
>  +1 There are no new bugs found in [sharelib/sqoop].
>  +1 There are no new bugs found in [sharelib/hive2].
>  +1 There are no new bugs found in [sharelib/git].
>  +1 There are no new bugs found in [sharelib/streaming].
> {noformat}
> Executing the same test locally found one error:
> {noformat}+1 PATCH_APPLIES
> 0 There are [1] new bugs found in total that would be nice to have fixed.
>  +1 There are no new bugs found in [tools].
>  0 There are [1] new bugs found in [core] that would be nice to have fixed.
>  You can find the FindBugs diff here: core/findbugs-new.html
>  +1 There are no new bugs found in [webapp].
>  +1 There are no new bugs found in [fluent-job/fluent-job-api].
>  +1 There are no new bugs found in [server].
>  +1 There are no new bugs found in [docs].
>  +1 There are no new bugs found in [sharelib/pig].
>  +1 There are no new bugs found in [sharelib/sqoop].
>  +1 There are no new bugs found in [sharelib/hcatalog].
>  +1 There are no new bugs found in [sharelib/hive2].
>  +1 There are no new bugs found in [sharelib/streaming].
>  +1 There are no new bugs found in [sharelib/oozie].
>  +1 There are no new bugs found in [sharelib/hive].
>  +1 There are no new bugs found in [sharelib/distcp].
>  +1 There are no new bugs found in [sharelib/git].
>  +1 There are no new bugs found in [sharelib/spark].
>  +1 There are no new bugs found in [examples].
>  +1 There are no new bugs found in [client].
> 
> +1 Overall result, good!, no -1s
> {noformat}
> Checking the output of the precommit shows that the patch is applied (at 
> least the number of classes are increased):
> {noformat}$ diff FINDBUGS_DIFF-{clean,patch}.txt
> 541c541
> < [INFO] Compiling 575 source files to 
> /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/core/target/classes
> ---
> > [INFO] Compiling 576 source files to 
> > /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/core/target/classes
> 568c568
> < [INFO] Compiling 392 source files to 
> /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/core/target/test-classes
> ---
> > [INFO] Compiling 393 source files to 
> > /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/core/target/test-classes
> ...
> {noformat}
> I've found one suspicious thing in the precommits logs:
> {noformat}[INFO] --- findbugs-maven-plugin:3.0.5:findbugs (findbugs) @ 
> oozie-core ---
> [INFO] Fork Value is true
>  [java] The following classes needed for analysis were missing:
>  [java]   getSharelibNames
>  [java] Warnings generated: 781
>  [java] Missing classes: 1
> [INFO] Done FindBugs Analysis
> {noformat}
> This {{getShareLibNames}} warning is not present in the local findbugs output.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3326 PreCommit Build #999

2019-02-15 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3326
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/999/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.91 MB...]
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [fluent-job/fluent-job-api].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[ERROR] There are [2] new bugs found below threshold in total that must be 
fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2995 bytes
[TRACE] Full summary file size is 1906 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3326

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [2] new bugs found below threshold in total that must be fixed.
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 -1 There are [2] new bugs found below threshold in [sharelib/oozie] that must 
be fixed.
 You can find the FindBugs diff here (look for the red and orange ones): 
sharelib/oozie/findbugs-new.html
 The most important FindBugs errors are:
 At LauncherAM.java:[line 95]: org.apache.oozie.action.hadoop.LauncherAM.sysenv 
isn't final but should be
 At LauncherMain.java:[line 99]: 
org.apache.oozie.action.hadoop.LauncherMain.sysenv isn't final but should be
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/git].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 3136
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/999/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  37190 0  100  3719  0   4803 --:--:-- --:--:-- --:--:--

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769222#comment-16769222
 ] 

Hadoop QA commented on OOZIE-3326:
--


Testing JIRA OOZIE-3326

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1{color} There are [2] new bugs found below threshold in total that 
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:red}-1{color} There are [2] new bugs found below threshold in 
[sharelib/oozie] that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones): 
sharelib/oozie/findbugs-new.html
. The most important FindBugs errors are:
. At LauncherAM.java:[line 95]: 
org.apache.oozie.action.hadoop.LauncherAM.sysenv isn't final but should be
. At LauncherMain.java:[line 99]: 
org.apache.oozie.action.hadoop.LauncherMain.sysenv isn't final but should be
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3136
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/999/



> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OZIE-3326_001.patch, 
> OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68317: Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Denes Bodo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68317/
---

(Updated Feb. 15, 2019, 12:33 p.m.)


Review request for oozie and András Piros.


Bugs: OOZIE-3326
https://issues.apache.org/jira/browse/OOZIE-3326


Repository: oozie-git


Description
---

SqoopMain needs to support tez delegation tokens for hive-imports. 
Implementation is similar to that of HiveMain and Hive2Main.

At present, hive-import will fail to start a tez session in secure environment.


Diffs (updated)
-

  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java 
63afd91d3 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
b6599f7f3 
  
sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SystemEnvironment.java
 PRE-CREATION 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 
27f9306a0 
  
sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopMain.java 
d6f96d546 


Diff: https://reviews.apache.org/r/68317/diff/3/

Changes: https://reviews.apache.org/r/68317/diff/2-3/


Testing
---


Thanks,

Denes Bodo

[jira] [Comment Edited] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769188#comment-16769188
 ] 

Andras Salamon edited comment on OOZIE-3326 at 2/15/19 10:54 AM:
-

Thanks [~dionusos] for the rebase. Can you please also update the modified 
patch to the Review Board.


was (Author: asalamon74):
Thasnk [~dionusos] for the rebase. Can you please also update the modified 
patch to the Review Board.

> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OZIE-3326_001.patch, 
> OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3431) Oozie web console serves image from http://extjs.com/s.gif

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769180#comment-16769180
 ] 

Andras Salamon commented on OOZIE-3431:
---

Thanks for the patch [~jtolar] it looks good to me. +1

I think we can ignore the testcase -1

> Oozie web console serves image from http://extjs.com/s.gif
> --
>
> Key: OOZIE-3431
> URL: https://issues.apache.org/jira/browse/OOZIE-3431
> Project: Oozie
>  Issue Type: Bug
>Reporter: Jacob Tolar
>Assignee: Jacob Tolar
>Priority: Minor
> Attachments: OOZIE-3431-001.patch
>
>
> When the Oozie web console is used, it serves an image from 
> [http://extjs.com/s.gif|http://extjs.com/s.gif.] . This should be fixed and 
> instead this file should be served from the Oozie host.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-3431) Oozie web console serves image from http://extjs.com/s.gif

2019-02-15 Thread Andras Salamon (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Salamon reassigned OOZIE-3431:
-

Assignee: Jacob Tolar

> Oozie web console serves image from http://extjs.com/s.gif
> --
>
> Key: OOZIE-3431
> URL: https://issues.apache.org/jira/browse/OOZIE-3431
> Project: Oozie
>  Issue Type: Bug
>Reporter: Jacob Tolar
>Assignee: Jacob Tolar
>Priority: Minor
> Attachments: OOZIE-3431-001.patch
>
>
> When the Oozie web console is used, it serves an image from 
> [http://extjs.com/s.gif|http://extjs.com/s.gif.] . This should be fixed and 
> instead this file should be served from the Oozie host.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows

2019-02-15 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769155#comment-16769155
 ] 

Andras Piros commented on OOZIE-2836:
-

[~kmarton] I think we can just remove the scripts, and maybe relevant parts of 
the documentation.

> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro 
> shell scripts were updated, but Windows scripts were not adjusted in 
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following 
> scripts: 
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd 
> - oozie.cmd 
> - oozied.cmd
> and verify that Oozie works on Windows.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69988: OOZIE-3395 Findbugs is no longer maintained

2019-02-15 Thread Andras Salamon 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69988/#review212832
---




bin/test-patch-11-findbugs-diff
Line 1 (original), 1 (patched)


Can you please check the script with shellcheck



bin/test-patch-11-findbugs-diff
Lines 28-29 (original), 28-29 (patched)


If we change the order of the variables, we don't need to enter the jar 
name twice.



bin/test-patch-11-findbugs-diff
Line 238 (original), 238 (patched)


Couldn't we use SPOTBUGS_XML_NAME here?



bin/test-patch-11-findbugs-diff
Line 275 (original), 275 (patched)


Can we change the name to spotbugs-new.xml? Or maybe it's hardwired 
somewhere else?



bin/test-patch-11-findbugs-diff
Line 331 (original), 331 (patched)


Even if we don't change the name, we could put it into a variable and reuse 
it here.



core/pom.xml
Lines 19 (patched)


Please delete dead code



fluent-job/fluent-job-api/pom.xml
Line 80 (original), 80 (patched)


Should we keep this filename?



fluent-job/pom.xml
Lines 44 (patched)


Do we really need the version number here? There is no version number in 
`core/pom.xml`



pom.xml
Lines 1632 (patched)


Please store the version number in a variable, so we can avoid repeating it 
multiple times.



pom.xml
Line 1798 (original), 1805 (patched)


Please store the version number in a variable, so we can avoid repeating it 
multiple times.



pom.xml
Line 1860 (original), 1866 (patched)


Please store the version number in a variable, so we can avoid repeating it 
multiple times.



pom.xml
Line 1865 (original), 1871 (patched)


Why is it called findbug? Compatibility reasons?



pom.xml
Line 1908 (original), 1914 (patched)


Please store the version number in a variable, so we can avoid repeating it 
multiple times.


- Andras Salamon


On Feb. 14, 2019, 4:51 p.m., Kinga Marton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69988/
> ---
> 
> (Updated Feb. 14, 2019, 4:51 p.m.)
> 
> 
> Review request for oozie and Andras Salamon.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://gleclaire.github.io/findbugs-maven-plugin/
> 
> Status: Since Findbugs is no longer maintained, please use Spotbugs which has 
> a Maven plugin.
> 
> The plugin author recommends to migrate to Spotbugs: 
> https://spotbugs.github.io/
> It might worth to investigate this plugin.
> 
> 
> Diffs
> -
> 
>   bin/test-patch-11-findbugs-diff c884daaa3 
>   client/pom.xml f0f6a1b13 
>   core/pom.xml b6c07d345 
>   fluent-job/fluent-job-api/pom.xml f303b4583 
>   fluent-job/pom.xml bb8861d59 
>   pom.xml 93fffc791 
>   sharelib/git/pom.xml 1c17e48ba 
>   sharelib/oozie/pom.xml a53d335f9 
>   sharelib/spark/pom.xml 76f69034e 
>   tools/pom.xml 72a6c283a 
> 
> 
> Diff: https://reviews.apache.org/r/69988/diff/1/
> 
> 
> Testing
> ---
> 
> Tested manually
> 
> 
> Thanks,
> 
> Kinga Marton
> 
>

[jira] [Commented] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769149#comment-16769149
 ] 

Andras Salamon commented on OOZIE-2836:
---

If the scripts were not updated two years ago and we had no Windows-specific 
bugreports then we don't really need to support Windows. 

> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro 
> shell scripts were updated, but Windows scripts were not adjusted in 
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following 
> scripts: 
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd 
> - oozie.cmd 
> - oozied.cmd
> and verify that Oozie works on Windows.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769140#comment-16769140
 ] 

Hadoop QA commented on OOZIE-3326:
--

PreCommit-OOZIE-Build started


> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OZIE-3326_001.patch, 
> OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Denes Bodo (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denes Bodo updated OOZIE-3326:
--
Attachment: OOZIE-3326_003.patch

> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OOZIE-3326_003.patch, OZIE-3326_001.patch, 
> OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows

2019-02-15 Thread Julia Kinga Marton (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769135#comment-16769135
 ] 

Julia Kinga Marton commented on OOZIE-2836:
---

Is there any need to make Oozie to work on Windows? If not, instead of fixing 
the scripts we might remove those.

[~asalamon74], [~andras.piros], [~gezapeti], what do you think?

> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro 
> shell scripts were updated, but Windows scripts were not adjusted in 
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following 
> scripts: 
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd 
> - oozie.cmd 
> - oozied.cmd
> and verify that Oozie works on Windows.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769132#comment-16769132
 ] 

Andras Salamon commented on OOZIE-3326:
---

[~dionusos] I will check the patch.

We need to rebase the patch first. [~bgoerlitz] can you rebase the patch?

> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769108#comment-16769108
 ] 

Hadoop QA commented on OOZIE-3326:
--

PreCommit-OOZIE-Build started


> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769119#comment-16769119
 ] 

Hadoop QA commented on OOZIE-3326:
--


Testing JIRA OOZIE-3326

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch




> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3326 PreCommit Build #998

2019-02-15 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3326
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/998/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 790.95 KB...]
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
oozie-zookeeper-security-tests ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/target/test-classes
[INFO] 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java:
 Some input files use or override a deprecated API.
[INFO] 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java:
 Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.5:check (default-cli) > :findbugs @ 
oozie-zookeeper-security-tests >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.5:findbugs (findbugs) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.5:check (default-cli) < :findbugs @ 
oozie-zookeeper-security-tests <<<
[INFO] 
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.5:check (default-cli) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] Reactor Summary for Apache Oozie Main 5.2.0-SNAPSHOT:
[INFO] 
[INFO] Apache Oozie Main .. SUCCESS [  2.211 s]
[INFO] Apache Oozie Fluent Job  SUCCESS [  0.051 s]
[INFO] Apache Oozie Fluent Job API  SUCCESS [ 25.132 s]
[INFO] Apache Oozie Client  SUCCESS [ 16.333 s]
[INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 12.765 s]
[INFO] Apache Oozie Share Lib HCatalog  SUCCESS [  6.097 s]
[INFO] Apache Oozie Share Lib Distcp .. SUCCESS [  5.034 s]
[INFO] Apache Oozie Core .. SUCCESS [01:22 min]
[INFO] Apache Oozie Share Lib Streaming ... SUCCESS [  8.285 s]
[INFO] Apache Oozie Share Lib Pig . SUCCESS [  8.227 s]
[INFO] Apache Oozie Share Lib Git . SUCCESS [ 12.870 s]
[INFO] Apache Oozie Share Lib Hive  SUCCESS [  8.414 s]
[INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [  9.247 s]
[INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [  6.149 s]
[INFO] Apache Oozie Examples .. SUCCESS [  9.224 s]
[INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 10.971 s]
[INFO] Apache Oozie Share Lib . SUCCESS [  0.019 s]
[INFO] Apache Oozie Docs .. SUCCESS [  3.281 s]
[INFO] Apache Oozie WebApp  SUCCESS [  9.073 s]
[INFO] Apache Oozie Tools . SUCCESS [ 14.064 s]
[INFO] Apache Oozie MiniOozie . SUCCESS [  1.717 s]
[INFO] Apache Oozie Fluent Job Client . SUCCESS [  0.460 s]
[INFO] Apache Oozie Server  SUCCESS [ 11.131 s]
[INFO] Apache Oozie Distro  SUCCESS [  2.304 s]
[INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [  3.012 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time:  04:30 min
[INFO] Finished at: 2019-02-15T09:18:42Z
[INFO] 
[TRACE] FindBugs output in HEAD verified and saved
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO

Applying patch

Checking patch 
a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java 
=> 
b/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java...
error: 
a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java: 
No such file or directory
Checking patch 
a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
=> 
b/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java...
error: 
a/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java:
 No such file or directory
Checking patch 
b/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SystemEnvironment.java...
Checking patch 
a/sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java =>

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Peter Cseh (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769114#comment-16769114
 ] 

Peter Cseh commented on OOZIE-3326:
---

Sorry, I won't have the time in the near future to check this one. Maybe 
[~asalamon74] will.


> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3326) Sqoop Action should support tez delegation tokens for hive-import

2019-02-15 Thread Denes Bodo (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769103#comment-16769103
 ] 

Denes Bodo commented on OOZIE-3326:
---

[~gezapeti] what do you think about this change? In production env the change 
has been tested. Thanks.

> Sqoop Action should support tez delegation tokens for hive-import
> -
>
> Key: OOZIE-3326
> URL: https://issues.apache.org/jira/browse/OOZIE-3326
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Brian Goerlitz
>Assignee: Brian Goerlitz
>Priority: Major
> Attachments: OZIE-3326_001.patch, OZIE-3326_002.patch
>
>
> SqoopMain needs to support tez delegation tokens for hive-imports. 
> Implementation is similar to that of HiveMain and Hive2Main.
> At present, hive-import will fail to start a tez session in secure 
> environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (OOZIE-2689) Spark options --keytab and --principal is not working from Spark Action

2019-02-15 Thread Julia Kinga Marton (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769088#comment-16769088
 ] 

Julia Kinga Marton edited comment on OOZIE-2689 at 2/15/19 8:57 AM:


It seems to be more Spark than Oozie related issue. It is handled by 
SPARK-21890.

 


was (Author: kmarton):
It seems to be more Spark than Oozie related issue. It seems to be fixed in 
SPARK-21890.

 

> Spark options --keytab and --principal is not working from Spark Action
> ---
>
> Key: OOZIE-2689
> URL: https://issues.apache.org/jira/browse/OOZIE-2689
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Priority: Major
>
> Spark job running longer than Kerberos ticket lifetime are failing because 
> the --principal and --keytab options are not working in Spark Action. 
> We're getting messages like
> {quote}
> Delegation Token can be issued only with kerberos or web authentication
> {quote}
> It's possible to work around these issue using the Shell Action and giving 
> these options to spark-submit, but it's not a good thing to do.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2689) Spark options --keytab and --principal is not working from Spark Action

2019-02-15 Thread Julia Kinga Marton (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769088#comment-16769088
 ] 

Julia Kinga Marton commented on OOZIE-2689:
---

It seems to be more Spark than Oozie related issue. It seems to be fixed in 
SPARK-21890.

 

> Spark options --keytab and --principal is not working from Spark Action
> ---
>
> Key: OOZIE-2689
> URL: https://issues.apache.org/jira/browse/OOZIE-2689
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Priority: Major
>
> Spark job running longer than Kerberos ticket lifetime are failing because 
> the --principal and --keytab options are not working in Spark Action. 
> We're getting messages like
> {quote}
> Delegation Token can be issued only with kerberos or web authentication
> {quote}
> It's possible to work around these issue using the Shell Action and giving 
> these options to spark-submit, but it's not a good thing to do.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3429) PreCommit FindBugs checking does not work

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769074#comment-16769074
 ] 

Andras Salamon commented on OOZIE-3429:
---

I don't really know why, but it was working for OOZIE-3418: 
https://builds.apache.org/job/PreCommit-OOZIE-Build/994/artifact/test-patch/reports/TEST-SUMMARY.txt

{noformat}-1 There are [2] new bugs found below threshold in total that must be 
fixed.
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/git].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [examples].
 -1 There are [2] new bugs found below threshold in [core] that must be fixed.
 You can find the FindBugs diff here (look for the red and orange ones): 
core/findbugs-new.html
 The most important FindBugs errors are:
 At FailingConnectionWrapper.java:[lines 357-373]: input must be non-null but 
is marked as nullable
 Method invoked at LiteWorkflowValidator.java:[line 185]: Null passed for 
non-null parameter of com.google.common.base.Objects.equal(Object, Object) in 
org.apache.oozie.workflow.lite.LiteWorkflowValidator.validateForkJoin(LiteWorkflowApp,
 NodeDef, NodeDef, String, boolean, Deque, Map, Map)
 Known null at LiteWorkflowValidator.java:[line 185]
{noformat}

> PreCommit FindBugs checking does not work
> -
>
> Key: OOZIE-3429
> URL: https://issues.apache.org/jira/browse/OOZIE-3429
> Project: Oozie
>  Issue Type: Bug
>Reporter: Andras Salamon
>Assignee: Julia Kinga Marton
>Priority: Major
>
> PreCommit FindBugs checking stopped reporting errors one or two months ago. 
> The first patch of OOZIE-3427 was 
> [reported|https://builds.apache.org/job/PreCommit-OOZIE-Build/992/consoleFull]
>  to be good:
> {noformat}+1 There are no new bugs found in total.
>  +1 There are no new bugs found in [client].
>  +1 There are no new bugs found in [server].
>  +1 There are no new bugs found in [fluent-job/fluent-job-api].
>  +1 There are no new bugs found in [webapp].
>  +1 There are no new bugs found in [examples].
>  +1 There are no new bugs found in [core].
>  +1 There are no new bugs found in [docs].
>  +1 There are no new bugs found in [tools].
>  +1 There are no new bugs found in [sharelib/distcp].
>  +1 There are no new bugs found in [sharelib/oozie].
>  +1 There are no new bugs found in [sharelib/hive].
>  +1 There are no new bugs found in [sharelib/spark].
>  +1 There are no new bugs found in [sharelib/pig].
>  +1 There are no new bugs found in [sharelib/hcatalog].
>  +1 There are no new bugs found in [sharelib/sqoop].
>  +1 There are no new bugs found in [sharelib/hive2].
>  +1 There are no new bugs found in [sharelib/git].
>  +1 There are no new bugs found in [sharelib/streaming].
> {noformat}
> Executing the same test locally found one error:
> {noformat}+1 PATCH_APPLIES
> 0 There are [1] new bugs found in total that would be nice to have fixed.
>  +1 There are no new bugs found in [tools].
>  0 There are [1] new bugs found in [core] that would be nice to have fixed.
>  You can find the FindBugs diff here: core/findbugs-new.html
>  +1 There are no new bugs found in [webapp].
>  +1 There are no new bugs found in [fluent-job/fluent-job-api].
>  +1 There are no new bugs found in [server].
>  +1 There are no new bugs found in [docs].
>  +1 There are no new bugs found in [sharelib/pig].
>  +1 There are no new bugs found in [sharelib/sqoop].
>  +1 There are no new bugs found in [sharelib/hcatalog].
>  +1 There are no new bugs found in [sharelib/hive2].
>  +1 There are no new bugs found in [sharelib/streaming].
>  +1 There are no new bugs found in [sharelib/oozie].
>  +1 There are no new bugs found in [sharelib/hive].
>  +1 There are no new bugs found in [sharelib/distcp].
>  +1 There are no new bugs found in [sharelib/git].
>  +1 There are no new bugs found in [sharelib/spark].
>  +1 There are no new bugs found in [examples].
>  +1 There are no new bugs found in [client].
> 
> +1 Overall result, good!, no -1s
> {noformat}
> Checking the output of the precommit shows that the patch is applied (at 
> least the number of

[jira] [Commented] (OOZIE-3395) Findbugs is no longer maintained

2019-02-15 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769053#comment-16769053
 ] 

Andras Salamon commented on OOZIE-3395:
---

PreCommit contains the following error message and the bugs session is missing 
from the summary.
{noformat}
bash: bin/test-patch-11-findbugs-diff: No such file or directory{noformat}

[~kmarton] Is it just temporary because of the findbugs - spotbugs switch?


> Findbugs is no longer maintained
> 
>
> Key: OOZIE-3395
> URL: https://issues.apache.org/jira/browse/OOZIE-3395
> Project: Oozie
>  Issue Type: Task
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-3395-001.patch
>
>
> https://gleclaire.github.io/findbugs-maven-plugin/
> {quote}
> Status: Since Findbugs is no longer maintained, please use Spotbugs which has 
> a Maven plugin.
> {quote}
> The plugin author recommends to migrate to Spotbugs: 
> https://spotbugs.github.io/
> It might  worth to investigate this plugin.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)