date:20181212

[jira] Subscription: Oozie Patch Available

2018-12-12 Thread jira

Issue Subscription
Filter: Oozie Patch Available (93 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3375  Can't use empty  in coordinator
https://issues.apache.org/jira/browse/OOZIE-3375
OOZIE-3367  Using && in EL expressions in oozie bundle.xml files generates 
parse errors
https://issues.apache.org/jira/browse/OOZIE-3367
OOZIE-3366  Update workflow status and subworkflow status on suspend command
https://issues.apache.org/jira/browse/OOZIE-3366
OOZIE-3364  Rerunning Oozie bundle jobs starts the coordinators in 
indeterminate order
https://issues.apache.org/jira/browse/OOZIE-3364
OOZIE-3362  When killed, SSH action should kill the spawned processes on target 
host
https://issues.apache.org/jira/browse/OOZIE-3362
OOZIE-3326  Sqoop Action should support tez delegation tokens for hive-import
https://issues.apache.org/jira/browse/OOZIE-3326
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3137  Add support for log4j2 in HiveMain
https://issues.apache.org/jira/browse/OOZIE-3137
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2784  Include WEEK as a parameter in the Coordinator Expression Language 
Evaulator
https://issues.apache.org/jira/browse/OOZIE-2784
OOZIE-2779  Mask Hive2 action Beeline JDBC password
https://issues.apache.org/jira/browse/OOZIE-2779
OOZIE-2736  Reduce the number of threads during test execution

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

2018-12-12 Thread Attila Sasvari

Hi there,

+1 (but please see my observations below)

Here is what I did:
- built Oozie with Hadoop version 2.7.5, installed Oozie sharelib, started
the server,
- executed example workflows using a single node, pseudo Hadoop cluster,
- clicked through the Oozie Web UI to validate basic functionality (checked
job statuses, logs, DAG),
- created a demo workflow using the Oozie Fluent job API, validated and
submitted via Oozie CLI,
- verified the issues reported on RC 1 are now fixed.

Some observations:
- I was hit by OOZIE-3378. As I understand that it will be included in the
next minor release.
- Some of the error messages contain error code twice. For example: "Error:
E0307 : E0307: Runtime error [App directory
[hdfs://localhost:9000/user/root/examples/apps/subwf] does not exist and
app definition cannot be created because of missing
config value [oozie.jobs.api.generated.xml]]"
- I noticed that when we validate workflows generated via the fluent job
API using the Oozie CLI with "-validatejar" and "--verbose" options,
temporary XML files are created in a world readable directory (/tmp) on the
client side. It is not a good thing, but I am not sure if it qualifies as a
blocker. What do you think? If we consider it a blocker, then OOZIE-3378
could be also included in the next RC.

- Attila


On Wed, Dec 12, 2018 at 5:15 PM Andras Salamon
 wrote:

> Thanks Peter,
>
> 1) I've opened an upstream jira:
> https://issues.apache.org/jira/browse/OOZIE-3401
>
> Sala
>
> On Wed, Dec 12, 2018 at 5:05 PM Andras Piros
>  wrote:
>
> > Thanks Peter,
> >
> > 2) resembles *PIG-5253 * which is in progress. Oozie's
> > Hadoop 3 support efforts are tracked under *OOZIE-2973
> > *.
> >
> > Regards,
> >
> > Andras
> >
> > On Wed, Dec 12, 2018 at 3:30 PM Peter Bacsko
>  > >
> > wrote:
> >
> > > +1 (binding, conditional)
> > >
> > > - verified hash
> > > - verified signature
> > > - built distro & ran all tests
> > > - created Hadoop pseudo dist cluster, executed sample Java workflow
> > >
> > > Things I observed:
> > > 1. TestPyspark constantly fails locally, but passes in Jenkins. Please
> > > investigate that it's a test failure and create a JIRA to track it.
> > > 2. Not possible to build distro if -Dhadoop.version is set to 2.9.2.
> > Please
> > > confirm that it's not Oozie's fault.
> > >
> > > Peter
> > >
> > >
> > > On Mon, Dec 10, 2018 at 1:50 PM Andras Piros 
> wrote:
> > >
> > > > Hi,
> > > >
> > > > as no votes were cast so far extending the deadline by one week,
> until
> > > > Friday, December 14, 2018 at 12:00am PDT.
> > > >
> > > > Please check the new RC and cast your vote accordingly.
> > > >
> > > > Thanks,
> > > >
> > > > Andras
> > > >
> > > >
> > > > On Mon, Dec 3, 2018 at 2:29 PM Andras Piros 
> wrote:
> > > >
> > > > > Hi there,
> > > > >
> > > > > I have created a build for Oozie 5.1.0, release candidate 2.
> > > > >
> > > > > This release introduces following major changes and fixes:
> > > > > - Fluent Job API
> > > > > - Git action
> > > > > - reduced heap waste
> > > > > - switched documentation from TWiki to Markdown
> > > > > - callable queue improvements
> > > > > - Spark action fixes
> > > > > - SLA improvements
> > > > > - third party dependency revamp
> > > > > - fixed CVE-2018-11799
> > > > >
> > > > > Keys to verify the signature of the release artifact are available
> at
> > > > >
> > > > >  http://www.apache.org/dist/oozie/KEYS
> > > > >
> > > > > Please download, test, and try it out:
> > > > >
> > > > > http://people.apache.org/~andras/oozie-5.1.0-rc2/
> > > > >
> > > > > The release candidate, sha512 signature, gpg signature, and rat
> > report
> > > > can
> > > > > all be found
> > > > > at the above address.
> > > > >
> > > > > The vote is open until Friday, December 7, 2018 at 12:00am PDT.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Andras Piros
> > > > >
> > > >
> > >
> >
>


-- 
-- 
Attila Sasvari
Software Engineer

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

2018-12-12 Thread Andras Salamon

Thanks Peter,

1) I've opened an upstream jira:
https://issues.apache.org/jira/browse/OOZIE-3401

Sala

On Wed, Dec 12, 2018 at 5:05 PM Andras Piros
 wrote:

> Thanks Peter,
>
> 2) resembles *PIG-5253 * which is in progress. Oozie's
> Hadoop 3 support efforts are tracked under *OOZIE-2973
> *.
>
> Regards,
>
> Andras
>
> On Wed, Dec 12, 2018 at 3:30 PM Peter Bacsko  >
> wrote:
>
> > +1 (binding, conditional)
> >
> > - verified hash
> > - verified signature
> > - built distro & ran all tests
> > - created Hadoop pseudo dist cluster, executed sample Java workflow
> >
> > Things I observed:
> > 1. TestPyspark constantly fails locally, but passes in Jenkins. Please
> > investigate that it's a test failure and create a JIRA to track it.
> > 2. Not possible to build distro if -Dhadoop.version is set to 2.9.2.
> Please
> > confirm that it's not Oozie's fault.
> >
> > Peter
> >
> >
> > On Mon, Dec 10, 2018 at 1:50 PM Andras Piros  wrote:
> >
> > > Hi,
> > >
> > > as no votes were cast so far extending the deadline by one week, until
> > > Friday, December 14, 2018 at 12:00am PDT.
> > >
> > > Please check the new RC and cast your vote accordingly.
> > >
> > > Thanks,
> > >
> > > Andras
> > >
> > >
> > > On Mon, Dec 3, 2018 at 2:29 PM Andras Piros  wrote:
> > >
> > > > Hi there,
> > > >
> > > > I have created a build for Oozie 5.1.0, release candidate 2.
> > > >
> > > > This release introduces following major changes and fixes:
> > > > - Fluent Job API
> > > > - Git action
> > > > - reduced heap waste
> > > > - switched documentation from TWiki to Markdown
> > > > - callable queue improvements
> > > > - Spark action fixes
> > > > - SLA improvements
> > > > - third party dependency revamp
> > > > - fixed CVE-2018-11799
> > > >
> > > > Keys to verify the signature of the release artifact are available at
> > > >
> > > >  http://www.apache.org/dist/oozie/KEYS
> > > >
> > > > Please download, test, and try it out:
> > > >
> > > > http://people.apache.org/~andras/oozie-5.1.0-rc2/
> > > >
> > > > The release candidate, sha512 signature, gpg signature, and rat
> report
> > > can
> > > > all be found
> > > > at the above address.
> > > >
> > > > The vote is open until Friday, December 7, 2018 at 12:00am PDT.
> > > >
> > > > Thanks,
> > > >
> > > > Andras Piros
> > > >
> > >
> >
>

[jira] [Created] (OOZIE-3401) TestPySpark failure

2018-12-12 Thread Andras Salamon (JIRA)

Andras Salamon created OOZIE-3401:
-

 Summary: TestPySpark failure
 Key: OOZIE-3401
 URL: https://issues.apache.org/jira/browse/OOZIE-3401
 Project: Oozie
  Issue Type: Bug
  Components: tests
Affects Versions: trunk
Reporter: Andras Salamon
 Fix For: 5.2.0


[TestPySpark|https://github.com/apache/oozie/blob/master/sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java]
 fails with the following error:

{noformat}[INFO] Running org.apache.oozie.action.hadoop.TestPyspark
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 131.97 
s <<< FAILURE! - in org.apache.oozie.action.hadoop.TestPyspark
[ERROR] testPyspark(org.apache.oozie.action.hadoop.TestPyspark)  Time elapsed: 
131.97 s  <<< FAILURE!
junit.framework.ComparisonFailure: expected:<[SUCCEED]ED> but 
was:<[FAILED/KILL]ED>
at junit.framework.Assert.assertEquals(Assert.java:100)
at junit.framework.Assert.assertEquals(Assert.java:107)
at junit.framework.TestCase.assertEquals(TestCase.java:269)
at 
org.apache.oozie.action.hadoop.TestPyspark.testPysparkHelper(TestPyspark.java:107)
at 
org.apache.oozie.action.hadoop.TestPyspark.testPyspark(TestPyspark.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176) 
at junit.framework.TestCase.runBare(TestCase.java:141) 
at junit.framework.TestResult$1.protect(TestResult.java:122)   
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125) 
at junit.framework.TestCase.run(TestCase.java:129) 
at junit.framework.TestSuite.runTest(TestSuite.java:255)   
at junit.framework.TestSuite.run(TestSuite.java:250)   
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26) 
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26) 
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at 
org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
at 
org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
at 
org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

  
at 
org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:593)
at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) 
  
at 
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
 
at 
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
at 
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
at 
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
at 
org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:159)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
   
at

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

2018-12-12 Thread Andras Piros

Thanks Peter,

2) resembles *PIG-5253 * which is in progress. Oozie's
Hadoop 3 support efforts are tracked under *OOZIE-2973
*.

Regards,

Andras

On Wed, Dec 12, 2018 at 3:30 PM Peter Bacsko 
wrote:

> +1 (binding, conditional)
>
> - verified hash
> - verified signature
> - built distro & ran all tests
> - created Hadoop pseudo dist cluster, executed sample Java workflow
>
> Things I observed:
> 1. TestPyspark constantly fails locally, but passes in Jenkins. Please
> investigate that it's a test failure and create a JIRA to track it.
> 2. Not possible to build distro if -Dhadoop.version is set to 2.9.2. Please
> confirm that it's not Oozie's fault.
>
> Peter
>
>
> On Mon, Dec 10, 2018 at 1:50 PM Andras Piros  wrote:
>
> > Hi,
> >
> > as no votes were cast so far extending the deadline by one week, until
> > Friday, December 14, 2018 at 12:00am PDT.
> >
> > Please check the new RC and cast your vote accordingly.
> >
> > Thanks,
> >
> > Andras
> >
> >
> > On Mon, Dec 3, 2018 at 2:29 PM Andras Piros  wrote:
> >
> > > Hi there,
> > >
> > > I have created a build for Oozie 5.1.0, release candidate 2.
> > >
> > > This release introduces following major changes and fixes:
> > > - Fluent Job API
> > > - Git action
> > > - reduced heap waste
> > > - switched documentation from TWiki to Markdown
> > > - callable queue improvements
> > > - Spark action fixes
> > > - SLA improvements
> > > - third party dependency revamp
> > > - fixed CVE-2018-11799
> > >
> > > Keys to verify the signature of the release artifact are available at
> > >
> > >  http://www.apache.org/dist/oozie/KEYS
> > >
> > > Please download, test, and try it out:
> > >
> > > http://people.apache.org/~andras/oozie-5.1.0-rc2/
> > >
> > > The release candidate, sha512 signature, gpg signature, and rat report
> > can
> > > all be found
> > > at the above address.
> > >
> > > The vote is open until Friday, December 7, 2018 at 12:00am PDT.
> > >
> > > Thanks,
> > >
> > > Andras Piros
> > >
> >
>

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

2018-12-12 Thread Peter Bacsko

+1 (binding, conditional)

- verified hash
- verified signature
- built distro & ran all tests
- created Hadoop pseudo dist cluster, executed sample Java workflow

Things I observed:
1. TestPyspark constantly fails locally, but passes in Jenkins. Please
investigate that it's a test failure and create a JIRA to track it.
2. Not possible to build distro if -Dhadoop.version is set to 2.9.2. Please
confirm that it's not Oozie's fault.

Peter


On Mon, Dec 10, 2018 at 1:50 PM Andras Piros  wrote:

> Hi,
>
> as no votes were cast so far extending the deadline by one week, until
> Friday, December 14, 2018 at 12:00am PDT.
>
> Please check the new RC and cast your vote accordingly.
>
> Thanks,
>
> Andras
>
>
> On Mon, Dec 3, 2018 at 2:29 PM Andras Piros  wrote:
>
> > Hi there,
> >
> > I have created a build for Oozie 5.1.0, release candidate 2.
> >
> > This release introduces following major changes and fixes:
> > - Fluent Job API
> > - Git action
> > - reduced heap waste
> > - switched documentation from TWiki to Markdown
> > - callable queue improvements
> > - Spark action fixes
> > - SLA improvements
> > - third party dependency revamp
> > - fixed CVE-2018-11799
> >
> > Keys to verify the signature of the release artifact are available at
> >
> >  http://www.apache.org/dist/oozie/KEYS
> >
> > Please download, test, and try it out:
> >
> > http://people.apache.org/~andras/oozie-5.1.0-rc2/
> >
> > The release candidate, sha512 signature, gpg signature, and rat report
> can
> > all be found
> > at the above address.
> >
> > The vote is open until Friday, December 7, 2018 at 12:00am PDT.
> >
> > Thanks,
> >
> > Andras Piros
> >
>

[jira] Subscription: Oozie Patch Available

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

[jira] [Created] (OOZIE-3401) TestPySpark failure

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

Re: [VOTE] Release Oozie 5.1.0 (candidate 2)

6 matches

Site Navigation

Mail list logo

Footer information