[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (93 issues) Subscriber: ooziedaily Key Summary OOZIE-3522 Migrate from Guava's Joiner https://issues.apache.org/jira/browse/OOZIE-3522 OOZIE-3506 Flaky test TestOozieRollingPolicy https://issues.apache.org/jira/browse/OOZIE-3506 OOZIE-3486 duplicate code in ControlNodeHandler https://issues.apache.org/jira/browse/OOZIE-3486 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3476 Migrate several classes to JUnit4 https://issues.apache.org/jira/browse/OOZIE-3476 OOZIE-3468 Use modernizer plugin https://issues.apache.org/jira/browse/OOZIE-3468 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib dire
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876160#comment-16876160 ] Hadoop QA commented on OOZIE-3522: -- Testing JIRA OOZIE-3522 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any star imports .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:red}-1{color} the patch does not add/modify any testcase {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} Javadoc generation succeeded with the patch .{color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [19] new bugs found below threshold in total that must be fixed. .{color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. .{color:red}-1{color} There are [15] new bugs found below threshold in [tools] that must be fixed, listing only the first [5] ones. .You can find the SpotBugs diff here (look for the red and orange ones): tools/findbugs-new.html .The top [5] most important SpotBugs errors are: .At OozieDBCLI.java:[line 584]: This use of java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL injection .At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573] .At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575] .At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578] .At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581] .{color:green}+1{color} There are no new bugs found in [sharelib/oozie]. .{color:green}+1{color} There are no new bugs found in [sharelib/streaming]. .{color:green}+1{color} There are no new bugs found in [sharelib/spark]. .{color:green}+1{color} There are no new bugs found in [sharelib/pig]. .{color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. .{color:green}+1{color} There are no new bugs found in [sharelib/git]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive]. .{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. .{color:green}+1{color} There are no new bugs found in [sharelib/distcp]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive2]. .{color:green}+1{color} There are no new bugs found in [server]. .{color:green}+1{color} There are no new bugs found in [examples]. .{color:red}-1{color} There are [4] new bugs found below threshold in [core] that must be fixed. .You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html .The most important SpotBugs errors are: .At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection .At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] .At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] .This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206] .At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127] .{color:green}+1{color} There are no new bugs found in [client]. .{color:green}+1{color} There are no new bugs found in [docs]. .{color:green}+1{color} There are no new bugs found in [webapp]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 3165 .{color:orange}Tests failed at first run:{color} TestCoordChangeXCommand#testCoordChangeEndTime3 .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/1159/ > Migrate fr
Failed: OOZIE-3522 PreCommit Build #1159
Jira: https://issues.apache.org/jira/browse/OOZIE-3522 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1159/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.03 MB...] [TRACE] Summary file size is 3965 bytes [TRACE] Full summary file size is 8189 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Testing JIRA OOZIE-3522 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 -1 the patch does not add/modify any testcase +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [19] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [fluent-job/fluent-job-api]. -1 There are [15] new bugs found below threshold in [tools] that must be fixed, listing only the first [5] ones. You can find the SpotBugs diff here (look for the red and orange ones): tools/findbugs-new.html The top [5] most important SpotBugs errors are: At OozieDBCLI.java:[line 584]: This use of java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL injection At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573] At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575] At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578] At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581] +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [examples]. -1 There are [4] new bugs found below threshold in [core] that must be fixed. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The most important SpotBugs errors are: At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206] At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127] +1 There are no new bugs found in [client]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [webapp]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 3165 Tests failed at first run: TestCoordChangeXCommand#testCoordChangeEndTime3 For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1159/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 48100 0 100 4810 0 4217 0:00:01 0:00:01 --:--:-- 4219{"self":"https://issues.apache.org/jira/rest/api/2/iss
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[ https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876151#comment-16876151 ] Andras Salamon commented on OOZIE-3468: --- Looking better: {noformat} $ mvn modernizer:modernizer | grep -i ERROR | sed -E 's/.*(Prefer .*)/\1/g' | sort | uniq -c | sort -nr 56 Prefer java.lang.StringBuilder 34 Prefer java.util.Collections.emptyList() 26 Prefer java.util.Collections.emptyMap() 26 Prefer java.lang.Long.valueOf(long) 21 Prefer java.lang.Integer.valueOf(int) 16 Prefer java.io.InputStreamReader(java.io.InputStream, java.nio.charset.Charset) 11 Prefer java.lang.String.(byte[], java.nio.charset.Charset) 6 Prefer java.lang.String.getBytes(java.nio.charset.Charset) 5 Prefer javax.inject.Provider 4 Prefer java.io.OutputStreamWriter(java.io.OutputStream, java.nio.charset.Charset) 3 Prefer java.util.LinkedHashSet<>() 2 Prefer java.util.LinkedHashMap<>() 1 Prefer java.util.HashMap<>() 1 Prefer java.util.Base64.Encoder.encodeToString(byte[]) 1 Prefer java.util.Base64.Decoder.decode(String) 1 Prefer java.lang.Double.valueOf(double){noformat} > Use modernizer plugin > - > > Key: OOZIE-3468 > URL: https://issues.apache.org/jira/browse/OOZIE-3468 > Project: Oozie > Issue Type: Improvement > Components: build >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3468-01-wip.patch > > > Recently I've opened a few jiras which suggested to use standard java classes > instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool > which can find such technical depts: [maven modernizer > plugin|https://github.com/gaul/modernizer-maven-plugin]. > The usage is quite simple: > {noformat} > $ mvn modernizer:modernizer > ... > [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88: > Prefer java.lang.String.(byte[], java.nio.charset.Charset) > > [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122: > Prefer java.nio.charset.StandardCharsets > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85: > Prefer java.nio.charset.StandardCharsets > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92: > Prefer java.nio.charset.StandardCharsets > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188: > Prefer java.util.ArrayList<>() > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91: > Prefer java.lang.StringBuilder > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101: > Prefer java.lang.StringBuilder > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110: > Prefer java.lang.StringBuilder > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119: > Prefer java.lang.StringBuilder > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36: > Prefer java.util.Collections.emptyList() > [ERROR] > /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49: > Prefer java.util.Collections.emptyList() > ... > {noformat} > There are several ways to use this plugin: > # Add the plugin to the root pom and let developers manually use this > plugin. It's the simplest solution, but it will be easy to forget it. > # Add this to the precommit script similarly to findbugs and at least avoid > to insert new code which uses old style API. Probably we will have the same > problems like we have with findbugs, we will have lots of false positive > warnings. > # Turn the plugin on by default and fail the compilation if it finds any > problem. I think this is too strict. > If we choose option 2 or 3 we should probably specify an ignore list, I don't > think for instance that we really want to change all the {{new Long(10)}} > code to {{Long.valueOf(10)}}. > By default this plugin checks the target java version (1.8 right now) but > it's possible to specify 1.7 instead if we want to focus on those problems > first. (It was not working for me without specifying the java version.) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3445) [build] Replace findbugs-diff jar
[ https://issues.apache.org/jira/browse/OOZIE-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876140#comment-16876140 ] Andras Salamon commented on OOZIE-3445: --- We should use the text output of spotbugs, as described here: [https://github.com/spotbugs/discuss/issues/55] > [build] Replace findbugs-diff jar > - > > Key: OOZIE-3445 > URL: https://issues.apache.org/jira/browse/OOZIE-3445 > Project: Oozie > Issue Type: Task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Minor > > During precommit, we use an extrenal[ > jar|https://repo1.maven.org/maven2/me/andrz/findbugs/findbugs-diff/0.1.0/]. > We should search for alternatives and replace it, because it not so nice to > store the md5, hardwiring the jar address and downloading the jar again and > again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876103#comment-16876103 ] Andras Salamon commented on OOZIE-3522: --- Something went wrong during the precommit, retriggering. > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876102#comment-16876102 ] Hadoop QA commented on OOZIE-3522: -- PreCommit-OOZIE-Build started > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Failed: OOZIE-3522 PreCommit Build #1158
Jira: https://issues.apache.org/jira/browse/OOZIE-3522 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1158/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.00 MB...] xargs: WARNING: a NUL character occurred in the input. It cannot be passed through in the argument list. Did you mean to use the --null option? Running test-patch task DISTRO Testing JIRA OOZIE-3522 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 -1 the patch does not add/modify any testcase +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [22] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [docs]. -1 There are [15] new bugs found below threshold in [tools] that must be fixed, listing only the first [5] ones. You can find the SpotBugs diff here (look for the red and orange ones): tools/findbugs-new.html The top [5] most important SpotBugs errors are: At OozieDBCLI.java:[line 584]: This use of java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL injection At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573] At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575] At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578] At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581] +1 There are no new bugs found in [fluent-job/fluent-job-api]. 0 There are [4] new bugs found in [server] that would be nice to have fixed. You can find the SpotBugs diff here: server/findbugs-new.html +1 There are no new bugs found in [webapp]. +1 There are no new bugs found in [examples]. -1 There are [7] new bugs found below threshold in [core] that must be fixed, listing only the first [5] ones. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The top [5] most important SpotBugs errors are: At BulkJPAExecutor.java:[line 207]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection At BulkJPAExecutor.java:[line 177]: At BulkJPAExecutor.java:[line 176] At BulkJPAExecutor.java:[line 206]: At BulkJPAExecutor.java:[line 200] This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 207] At BulkJPAExecutor.java:[line 112]: At BulkJPAExecutor.java:[line 128] +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files -1 TESTS Tests run: 2967 Tests failed : 1 Tests in error : 0 Tests timed out : 0 -1 [ERROR] There are [1] test failures in [core]. Listing only the first [5] ones testPurgeableWorkflowPurgeableSubWorkflowPurgeableSubSubWorkflow:org.apache.oozie.command.TestPurgeXCommand Check console output for the full list of errors/failures +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1158/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 51370 0 100 5137 0 4411 0:00:01 0:00:01 --:--:-- 4413{"self":"https://issues.apache.org/jira/rest/api
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876076#comment-16876076 ] Hadoop QA commented on OOZIE-3522: -- Testing JIRA OOZIE-3522 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any star imports .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:red}-1{color} the patch does not add/modify any testcase {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} Javadoc generation succeeded with the patch .{color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [22] new bugs found below threshold in total that must be fixed. .{color:green}+1{color} There are no new bugs found in [sharelib/hive2]. .{color:green}+1{color} There are no new bugs found in [sharelib/spark]. .{color:green}+1{color} There are no new bugs found in [sharelib/oozie]. .{color:green}+1{color} There are no new bugs found in [sharelib/pig]. .{color:green}+1{color} There are no new bugs found in [sharelib/streaming]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive]. .{color:green}+1{color} There are no new bugs found in [sharelib/distcp]. .{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. .{color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. .{color:green}+1{color} There are no new bugs found in [sharelib/git]. .{color:green}+1{color} There are no new bugs found in [client]. .{color:green}+1{color} There are no new bugs found in [docs]. .{color:red}-1{color} There are [15] new bugs found below threshold in [tools] that must be fixed, listing only the first [5] ones. .You can find the SpotBugs diff here (look for the red and orange ones): tools/findbugs-new.html .The top [5] most important SpotBugs errors are: .At OozieDBCLI.java:[line 584]: This use of java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL injection .At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573] .At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575] .At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578] .At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581] .{color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. .{color:orange}0{color} There are [4] new bugs found in [server] that would be nice to have fixed. .You can find the SpotBugs diff here: server/findbugs-new.html .{color:green}+1{color} There are no new bugs found in [webapp]. .{color:green}+1{color} There are no new bugs found in [examples]. .{color:red}-1{color} There are [7] new bugs found below threshold in [core] that must be fixed, listing only the first [5] ones. .You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html .The top [5] most important SpotBugs errors are: .At BulkJPAExecutor.java:[line 207]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection .At BulkJPAExecutor.java:[line 177]: At BulkJPAExecutor.java:[line 176] .At BulkJPAExecutor.java:[line 206]: At BulkJPAExecutor.java:[line 200] .This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 207] .At BulkJPAExecutor.java:[line 112]: At BulkJPAExecutor.java:[line 128] {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2967 .Tests failed : 1 .Tests in error : 0 .Tests timed out : 0 {color:red}-1{color} [ERROR] There are [1] test failures in [core]. Listing only the first [5] ones testPurgeableWorkflowPurgeableSubWorkflowPurgeableSubSubWorkflow:org.apache.oozie.command.TestPurgeXCommand Check console output for the full list of errors/failures {color
[jira] [Commented] (OOZIE-3520) Upgrade Derby to 10.14.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876050#comment-16876050 ] Andras Salamon commented on OOZIE-3520: --- Thanks for the contribution [~nobigo], +1, committed to master. > Upgrade Derby to 10.14.2.0 > -- > > Key: OOZIE-3520 > URL: https://issues.apache.org/jira/browse/OOZIE-3520 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: duan xiong >Priority: Major > Attachments: OOZIE-3520-001.patch > > > Upgrade Derby to the latest in the 10.14.x line: 10.14.2.0. > We cannot upgrade to 10.15, since it requires JDK9: > [https://db.apache.org/derby/derby_downloads.html] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[ https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876045#comment-16876045 ] Julia Kinga Marton commented on OOZIE-3513: --- Thank you @asalamon for the contribution! +1, committed to master > Migrate from Preconditions.checkNotNull and ParamChecker.notNull > > > Key: OOZIE-3513 > URL: https://issues.apache.org/jira/browse/OOZIE-3513 > Project: Oozie > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch, > OOZIE-3513-03.patch, OOZIE-3513-04.patch, OOZIE-3513-05.patch > > > We currently use both Guava's {{Preconditions.checkNotNull}} and our own > {{ParamChecker.notNull}} to check for null arguments. Instead we should use > the standard {{Objects.requireNonNull}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70964: OOZIE-3513 Migrate from Preconditions.checkNotNull and ParamChecker.notNull
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70964/#review216267 --- Ship it! Ship It! - Kinga Marton On June 28, 2019, 8:57 a.m., Andras Salamon wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70964/ > --- > > (Updated June 28, 2019, 8:57 a.m.) > > > Review request for oozie and Kinga Marton. > > > Repository: oozie-git > > > Description > --- > > OOZIE-3513 Migrate from Preconditions.checkNotNull and ParamChecker.notNull > > > Diffs > - > > client/src/main/java/org/apache/oozie/client/ApiJarFactory.java bb85d6780 > client/src/main/java/org/apache/oozie/client/ApiJarLoader.java 6ecf485ed > core/src/main/java/org/apache/oozie/BaseLocalOozieClient.java 376ea1165 > core/src/main/java/org/apache/oozie/XException.java fb1a381bb > core/src/main/java/org/apache/oozie/action/ActionExecutorException.java > 9bc0e81b9 > core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java > fc4d65b83 > core/src/main/java/org/apache/oozie/action/hadoop/HadoopTokenHelper.java > 1bdeb32f6 > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > ec45fe495 > > core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java > a509e4d98 > core/src/main/java/org/apache/oozie/action/hadoop/ShareLibExcluder.java > 9c45e5392 > core/src/main/java/org/apache/oozie/command/TransitionXCommand.java > 262a78f5b > > core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java > bc58be6ef > > core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java > ab88cac6c > > core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java > d51f0d780 > > core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java > ef0fc2d3e > > core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java > 9646d738a > > core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java > 88842f1f1 > core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java > 2f158cdbf > core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java > 2622f05d9 > core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java dd8d3d2f6 > core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java > 9ee84063a > core/src/main/java/org/apache/oozie/command/wf/SubmitXCommand.java > 70b9adc1c > > core/src/main/java/org/apache/oozie/command/wf/WorkflowNotificationXCommand.java > 62bf9b5b3 > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 2761e44b0 > core/src/main/java/org/apache/oozie/coord/CoordUtils.java 1d97accd4 > core/src/main/java/org/apache/oozie/executor/jpa/BulkJPAExecutor.java > 42961157a > > core/src/main/java/org/apache/oozie/executor/jpa/BundleActionGetJPAExecutor.java > 3293efd69 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleActionInsertJPAExecutor.java > 4b99c1a59 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleActionsCountForJobGetJPAExecutor.java > c9f3a212d > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobGetCoordinatorsJPAExecutor.java > 2f4665e15 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobGetForUserJPAExecutor.java > aef9eb79c > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobGetJPAExecutor.java > 7f98cd2a8 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobInfoGetJPAExecutor.java > 6e1ea3e5b > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobInsertJPAExecutor.java > e7373d67d > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForCheckJPAExecutor.java > 9e8b33191 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForExternalIdJPAExecutor.java > 61614e9fa > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInfoJPAExecutor.java > 211d594e5 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java > 484ff93c9 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForStartJPAExecutor.java > 14b831d65 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForTimeoutJPAExecutor.java > c720be99d > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetJPAExecutor.java > 53214ea64 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionInsertJPAExecutor.java > 09e885793 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionRemoveJPAExecutor.java > 8ea31a31d > > core/src/main/java/org/apache/ooz
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876023#comment-16876023 ] Hadoop QA commented on OOZIE-3522: -- PreCommit-OOZIE-Build started > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3522) Migrate from Guava's Joiner
Andras Salamon created OOZIE-3522: - Summary: Migrate from Guava's Joiner Key: OOZIE-3522 URL: https://issues.apache.org/jira/browse/OOZIE-3522 Project: Oozie Issue Type: Sub-task Reporter: Andras Salamon Assignee: Andras Salamon As mentioned in OOZIE-3488 we might use standard JDK classes instead of the [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3488) Migrate from guava classes to the base Java implementations
[ https://issues.apache.org/jira/browse/OOZIE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-3488: -- Issue Type: Sub-task (was: Improvement) Parent: OOZIE-3521 > Migrate from guava classes to the base Java implementations > --- > > Key: OOZIE-3488 > URL: https://issues.apache.org/jira/browse/OOZIE-3488 > Project: Oozie > Issue Type: Sub-task >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: code-cleanup > Fix For: 5.2.0 > > Attachments: OOZIE-3488-2.patch, OOZIE-3488-3.patch, OOZIE-3488.patch > > > In Oozie, guava classes are used even when Java 8 already provides a > compatible implementation, removing these usages would reduce the dependency > to guava (which is regularly a headache due to her backward incompatible > changes) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[ https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-3513: -- Issue Type: Sub-task (was: Bug) Parent: OOZIE-3521 > Migrate from Preconditions.checkNotNull and ParamChecker.notNull > > > Key: OOZIE-3513 > URL: https://issues.apache.org/jira/browse/OOZIE-3513 > Project: Oozie > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch, > OOZIE-3513-03.patch, OOZIE-3513-04.patch, OOZIE-3513-05.patch > > > We currently use both Guava's {{Preconditions.checkNotNull}} and our own > {{ParamChecker.notNull}} to check for null arguments. Instead we should use > the standard {{Objects.requireNonNull}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3467) Migrate from com.google.common.base.Stopwatch
[ https://issues.apache.org/jira/browse/OOZIE-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-3467: -- Issue Type: Sub-task (was: Improvement) Parent: OOZIE-3521 > Migrate from com.google.common.base.Stopwatch > - > > Key: OOZIE-3467 > URL: https://issues.apache.org/jira/browse/OOZIE-3467 > Project: Oozie > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3467-01.patch > > > OOZIE-1826 created {{JvmPauseMonitorService}} which uses Guava's stopwatch. > The implementation was heavily inspired by HADOOP-9618. Guava changed the > StopWatch class in a way that we either create a version which is compatible > in the old (Guava 11) or the new (Guava 27) interface. To make the code Guava > version independent we should not use this class anymore. Hadoop also > replaced this class in HADOOP-11032. > We don't really need nanosecond accuracy here, we only read milliseconds from > the stopwatch and the default thresholds are 1 second > (oozie.service.JvmPauseMonitorService.info-threshold.ms = 1000ms) and 10 > seconds (oozie.service.JvmPauseMonitorService.warn-threshold.ms = 10 000ms) > so even millisecond is a bit overkill. We could just use > {{System.currentTimeMillis()}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3466) Migrate from com.google.common.io.Closeables to org.apache.commons.io.IOUtils
[ https://issues.apache.org/jira/browse/OOZIE-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-3466: -- Issue Type: Sub-task (was: Improvement) Parent: OOZIE-3521 > Migrate from com.google.common.io.Closeables to org.apache.commons.io.IOUtils > - > > Key: OOZIE-3466 > URL: https://issues.apache.org/jira/browse/OOZIE-3466 > Project: Oozie > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3466-01.patch > > > [JavaActionExecutor|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java] > uses both Guava's Closeables and commons-io's IOUtils to close the > {{yarnClient}} quietly. The Guava's versions causes problem in the Guava > update (OOZIE-3418). We should use only commons-io's IOUtils in the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3521) [Umbrella] Reduce Guava usage
Andras Salamon created OOZIE-3521: - Summary: [Umbrella] Reduce Guava usage Key: OOZIE-3521 URL: https://issues.apache.org/jira/browse/OOZIE-3521 Project: Oozie Issue Type: Bug Reporter: Andras Salamon In the last weeks we have resolved a few jiras which reduced the guava usage. Let's create an umbrella Jira to collect these jiras and discuss further ideas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)