[jira] [Commented] (OOZIE-3449) Make spark-2 as the default profile
[
https://issues.apache.org/jira/browse/OOZIE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795585#comment-16795585
]
Hadoop QA commented on OOZIE-3449:
--
Testing JIRA OOZIE-3449
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:red}-1{color} There are [1] new bugs found below threshold in
[sharelib/oozie] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
sharelib/oozie/findbugs-new.html
.The most important SpotBugs errors are:
.At ShellMain.java:[line 92]: This usage of
java/lang/ProcessBuilder.(Ljava/util/List;)V can be vulnerable to
Command Injection
.At ShellMain.java:[line 90]: At ShellMain.java:[line 89]
.At ShellMain.java:[line 91]
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:orange}0{color} There are [4] new bugs found in [server] that would
be nice to have fixed.
.You can find the SpotBugs diff here: server/findbugs-new.html
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3169
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1048/
> Make spark-2 as the default profile
> ---
>
> Key: OOZIE-3449
> URL: https://issues.apache.org/jira/browse/OOZIE-3449
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: duan xiong
>Priority: Major
> Attachments: OOZIE-3449-001.patch
>
>
> OOZIE-2606 introduced spark-1 and spark-2 profiles more than 2 years ago.
> Right now spark-1 profile is the default. I think it's time to make the
> spark-2 profile as default.
> We need to check the unit tests and the demos before we change the default.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3449 PreCommit Build #1048
Jira: https://issues.apache.org/jira/browse/OOZIE-3449 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1048/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 1.95 MB...] [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [core]. [ERROR] There are [1] new bugs found below threshold in total that must be fixed. [TRACE] SpotBugs diffs checked and reports created [TRACE] Summary file size is 3099 bytes [TRACE] Full summary file size is 2088 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Unregistered error message /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Premature end of data in tag system-out line 139 /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Premature end of data in tag testcase line 89 /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Premature end of data in tag testsuite line 2 /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Unregistered error message /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Premature end of data in tag system-out line 139 /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Premature end of data in tag testcase line 89 /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/TESTS/results/sharelib/spark/target/surefire-reports/TEST-org.apache.oozie.action.hadoop.TestPyspark.xml:139: parser error : Premature end of data in tag testsuite line 2 Running test-patch task DISTRO Testing JIRA OOZIE-3449 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 -1 the patch does not add/modify any testcase +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [1] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/spark]. -1 There are [1] new bugs found below threshold in [sharelib/oozie] that must be fixed. You can find the SpotBugs diff here (look for the red and orange ones): sharelib/oozie/findbugs-new.html The most important SpotBugs errors are: At ShellMain.java:[line 92]: This usage of java/lang/ProcessBuilder.(Ljava/util/List;)V can be vulnerable to Command Injection At ShellMain.java:[line 90]: At ShellMain.java:[line 89] At ShellMain.java:[line 91] +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [tools]. +1
[jira] [Commented] (OOZIE-3449) Make spark-2 as the default profile
[ https://issues.apache.org/jira/browse/OOZIE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795530#comment-16795530 ] Hadoop QA commented on OOZIE-3449: -- PreCommit-OOZIE-Build started > Make spark-2 as the default profile > --- > > Key: OOZIE-3449 > URL: https://issues.apache.org/jira/browse/OOZIE-3449 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: duan xiong >Priority: Major > Attachments: OOZIE-3449-001.patch > > > OOZIE-2606 introduced spark-1 and spark-2 profiles more than 2 years ago. > Right now spark-1 profile is the default. I think it's time to make the > spark-2 profile as default. > We need to check the unit tests and the demos before we change the default. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3449) Make spark-2 as the default profile
[ https://issues.apache.org/jira/browse/OOZIE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795077#comment-16795077 ] duan xiong commented on OOZIE-3449: --- Hi,[~asalamon74],I have check the unit tests and the demos.It is compatible with Spark2.1.0. > Make spark-2 as the default profile > --- > > Key: OOZIE-3449 > URL: https://issues.apache.org/jira/browse/OOZIE-3449 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: duan xiong >Priority: Major > Attachments: OOZIE-3449-001.patch > > > OOZIE-2606 introduced spark-1 and spark-2 profiles more than 2 years ago. > Right now spark-1 profile is the default. I think it's time to make the > spark-2 profile as default. > We need to check the unit tests and the demos before we change the default. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V1.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: (was: OOZIE-3443-V1.patch) > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V1.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz reassigned OOZIE-3443: -- Assignee: Mate Juhasz > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3449) Make spark-2 as the default profile
[ https://issues.apache.org/jira/browse/OOZIE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duan xiong updated OOZIE-3449: -- Attachment: OOZIE-3449-001.patch > Make spark-2 as the default profile > --- > > Key: OOZIE-3449 > URL: https://issues.apache.org/jira/browse/OOZIE-3449 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: duan xiong >Priority: Major > Attachments: OOZIE-3449-001.patch > > > OOZIE-2606 introduced spark-1 and spark-2 profiles more than 2 years ago. > Right now spark-1 profile is the default. I think it's time to make the > spark-2 profile as default. > We need to check the unit tests and the demos before we change the default. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3451) Example in Git action's documentation refers to incorrect worflow version
Attila Sasvari created OOZIE-3451:
-
Summary: Example in Git action's documentation refers to incorrect
worflow version
Key: OOZIE-3451
URL: https://issues.apache.org/jira/browse/OOZIE-3451
Project: Oozie
Issue Type: Bug
Components: docs
Reporter: Attila Sasvari
https://oozie.apache.org/docs/5.1.0/DG_GitActionExtension.html uses an example
with
{{}}
That is incorrect as per:
https://github.com/apache/oozie/blob/master/client/src/main/resources/git-action-1.0.xsd
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (87 issues) Subscriber: ooziedaily Key Summary OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3249 [tools] Instrumentation log parser https://issues.apache.org/jira/browse/OOZIE-3249 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator https://issues.apache.org/jira/browse/OOZIE-2784 OOZIE-2779 Mask Hive2 action Beeline JDBC password https://issues.apache.org/jira/browse/OOZIE-2779 OOZIE-2736 Reduce the number of threads during test execution https://issues.apache.org/jira/browse/OOZIE-2736 OOZIE-2714 Detect conflicting resources during class loading https://issues.apache.org/jira/browse/OOZIE-2714 OOZIE-2692 Oozie job submit doesn't report error message to user if there is any issue with job conf
