[jira] [Commented] (OOZIE-3601) Upgrade quartz to 2.3.2
[ https://issues.apache.org/jira/browse/OOZIE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17154424#comment-17154424 ] Mate Juhasz commented on OOZIE-3601: Thanks [~dengliming] for the contribution, committed to master! > Upgrade quartz to 2.3.2 > --- > > Key: OOZIE-3601 > URL: https://issues.apache.org/jira/browse/OOZIE-3601 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.0 >Reporter: Andras Salamon >Assignee: dengliming >Priority: Major > Attachments: OOZIE-3601-001.patch > > > We should upgrade quartz to 2.3.2: > https://nvd.nist.gov/vuln/detail/CVE-2019-13990 > Most likely it's just a simple version bump. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (OOZIE-3599) Upgrade Jetty to 9.4.28
Mate Juhasz created OOZIE-3599: -- Summary: Upgrade Jetty to 9.4.28 Key: OOZIE-3599 URL: https://issues.apache.org/jira/browse/OOZIE-3599 Project: Oozie Issue Type: Bug Affects Versions: 5.2.0 Reporter: Mate Juhasz Fix For: trunk Jetty could be upgraded to the latest version, which is now 9.4.28.v20200408, though it has to be checked before if its feasible. In case we upgrade we have to be aware of wildcard certs with SAN as well and should change the code to use SslContextFactory.server instead of SsslContextFactory in org.apache.oozie.server.SSLServerConnectorFactory class. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3575:
---
Description:
Oozie by default gathers delegation tokens for the nodes defined in
_mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_ in
case of distcp actions) and for the workflow path.
Though this implementation is good for hdfs, we dont support occasions where
the job related resources, which we want to access in runtime are present on
different file systems/buckets etc...
The HDFSCredentials class should be revised to handle getting tokens for
different cloud storages.
*The following scenarios should be addressed:*
Oozie should obtain delegation token in case
* the defaultFs is cloud
* the workload.xml is in cloud
* input/output/auxiliary files referred from workflow are in cloud
* (newly introduced feature) user could define filesystem credentials for the
workflow (as its done with hive/hcat etc..) -> this would allow the user to
handle the situation where Oozie could not decide which tokens are needed at
launch time by default and could also get tokens for different cloud storages
and buckets as well
Example for credentials addition:
{noformat}
filesystem.path
s3a://qe-s3-bucket-mst
{noformat}
was:
Oozie by default gathers delegation tokens for the nodes defined in
_mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_ in
case of distcp actions) and for the workflow path.
Though this implementation is good for hdfs, we dont support occasions where
the job related resources, which we want to access in runtime are present on
different file systems/buckets etc...
The HDFSCredentials class should be revised to handle getting tokens for
different cloud storages.
*The following scenarios should be addressed:*
Oozie should obtain delegation token in case
* the defaultFs is cloud
* the workload.xml is in cloud
* input/output/auxiliary files referred from workflow are in cloud
* (newly introduced feature) user could define filesystem credentials for the
workflow (as its done with hive/hcat etc..) -> this would allow the user to
handle the situation where Oozie could not decide which tokens are needed at
launch time by default and could also get tokens for different cloud storages
and buckets as well
Example for credentials addition:
{noformat}
filesystem
s3a://qe-s3-bucket-mst
{noformat}
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.2.0
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3575-v2.patch, OOZIE-3575-v3.patch,
> OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem.path
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: OOZIE-3592-v3.patch
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch,
> OOZIE-3592-v3.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: (was: OOZIE-3592-v2.patch)
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: OOZIE-3592-v2.patch
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: OOZIE-3592-v2.patch
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053394#comment-17053394
]
Mate Juhasz commented on OOZIE-3592:
Added an initial patch, but I still owe with unit tests
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: OOZIE-3592-v1.patch
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Summary: Do not print misleading System.exit() exception for successful
jobs (was: Do not print misleading security exception for successful jobs)
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(1)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (OOZIE-3592) Do not print misleading security exception for successful jobs
Mate Juhasz created OOZIE-3592:
--
Summary: Do not print misleading security exception for successful
jobs
Key: OOZIE-3592
URL: https://issues.apache.org/jira/browse/OOZIE-3592
Project: Oozie
Issue Type: Bug
Affects Versions: 5.2.0
Reporter: Mate Juhasz
Assignee: Mate Juhasz
Fix For: trunk
The following exception is logged by Oozie due to System.exit() calls in
application JVM-s even after successful workflow actions:
{noformat}
Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
at
org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
Caused by: java.lang.SecurityException: Intercepted System.exit(1)
at
org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
at java.lang.Runtime.exit(Runtime.java:107)
at java.lang.System.exit(System.java:971)
at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
at
org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
at
org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
... 16 more
Intercepting System.exit(0)
{noformat}
Typically happens in Sqoop actions. This can be very misleading for users as
they may suspect that some error has occurred even if their yarn application
and Oozie workflow has been marked as succeeded.
The "error" is already ignored by Oozie when the System.exit code is 0, but we
should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3584) Fork-join action issue when action param cannot be resolved
[ https://issues.apache.org/jira/browse/OOZIE-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3584: --- Fix Version/s: trunk > Fork-join action issue when action param cannot be resolved > --- > > Key: OOZIE-3584 > URL: https://issues.apache.org/jira/browse/OOZIE-3584 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.1.0 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Fix For: trunk > > Attachments: OOZIE-3584-001.patch, forkjoin_actionparam_issue.log > > > _Current bug_ > *=* > There is a sub workflow run in independent mode that runs a fork action which > contains two (or more) actions. > These actions inside the fork action run in parallel mode, and they have some > seconds delay in between them. > If a parameter is passed to one of these actions, that cannot be resolved, > then it changes its status to FAILED, and also the workflow’s state to > FAILED. The other action’s state which are not started yet will stuck in PREP > state forever. The correct behaviour would be to KILL the remaining actions > as well as the workflow. > Note: this bug only occurs when it is run in independent mode. If it has a > parent workflow, then the parent workflow will kill this workflow after 10 > minutes because of the callback process. > > _Log_ > *===* > 2020-02-14 11:59:26,698 ERROR org.apache.oozie.command.wf.SignalXCommand: > SERVER[quasar-nqrrjp-4.quasar-nqrrjp.root.hwx.site] USER[admin] GROUP[-] > TOKEN[] APP[Sub flow fork join] JOB[005-200214101441478-oozie-oozi-W] > ACTION[005-200214101441478-oozie-oozi-W@fork-4a1c] Error running forked > jobs parallely > org.apache.oozie.command.CommandException: E0718: Workflow already completed > at > org.apache.oozie.command.wf.ActionXCommand.failJob(ActionXCommand.java:213) > at > org.apache.oozie.command.wf.ActionXCommand.failJob(ActionXCommand.java:185) > at > org.apache.oozie.command.wf.SignalXCommand.startForkedActions(SignalXCommand.java:498) > at > org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:462) > at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:82) > at org.apache.oozie.command.XCommand.call(XCommand.java:291) > at > org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:283) > at > org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:62) > at org.apache.oozie.command.XCommand.call(XCommand.java:291) > at > org.apache.oozie.command.wf.ActionStartXCommand.callActionEnd(ActionStartXCommand.java:352) > at > org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:338) > at > org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:68) > at org.apache.oozie.command.XCommand.call(XCommand.java:291) > at > org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:363) > at > org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:292) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > Caused by: org.apache.oozie.workflow.WorkflowException: E0718: Workflow > already completed > at > org.apache.oozie.workflow.lite.LiteWorkflowInstance.fail(LiteWorkflowInstance.java:337) > at > org.apache.oozie.command.wf.ActionXCommand.failJob(ActionXCommand.java:201) > ... 19 more > Full log added as attachment. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031477#comment-17031477
]
Mate Juhasz commented on OOZIE-3071:
Bumping up the topic again... commons-lang3 3.3.2 does not support JDK 11, the
below error can be observed when running spark actions on Java 11.
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(SystemUtils.java:1654)
{noformat}
commons-lang3:3.8.1 fixes it, but we should use the same 3.9 as in SPARK-30035,
which is the first release to support JDK9+.
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
>Assignee: Artem Ervits
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3580) setLibFilesArchives is called twice unnecessarily
[
https://issues.apache.org/jira/browse/OOZIE-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017024#comment-17017024
]
Mate Juhasz commented on OOZIE-3580:
Ok, so its really not safe to drop any of these method calls. There is some
magic done with configuration files, I need to check it further
> setLibFilesArchives is called twice unnecessarily
> -
>
> Key: OOZIE-3580
> URL: https://issues.apache.org/jira/browse/OOZIE-3580
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
>
> The JavaActionExecutor#setLibFilesArchives method can be costly as its
> iterating through all the files, archives, lib files and share lib files
> added for an action. Its invoked twice per workflow at the moment, but the
> second call does make any difference.
> {code:java}
> public void setLibFilesArchives(Context context, Element actionXml, Path
> appPath, Configuration conf)
> throws ActionExecutorException {
> addWfApplicationLibs(appPath, conf, context.getProtoActionConf());
> addActionXmlFilesAndArchives(actionXml, appPath, conf);
> initShareLibExcluder(conf, context);
> addActionLibs(appPath, conf);
> addAllShareLibs(appPath, conf, context, actionXml);
> }{code}
> First setLibFilesArchives is invoked in JavaActionExecutor#submitLauncher and
> then submitLauncher calls createLauncherConf, where its invoked again, but
> one of them is unnecessary.
> I tested around a bit and its probably safe to remove the setLibFilesArchives
> call for example from submitLauncher.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (OOZIE-3580) setLibFilesArchives is called twice unnecessarily
Mate Juhasz created OOZIE-3580:
--
Summary: setLibFilesArchives is called twice unnecessarily
Key: OOZIE-3580
URL: https://issues.apache.org/jira/browse/OOZIE-3580
Project: Oozie
Issue Type: Bug
Components: core
Affects Versions: 5.2.0
Reporter: Mate Juhasz
Assignee: Mate Juhasz
Fix For: trunk
The JavaActionExecutor#setLibFilesArchives method can be costly as its
iterating through all the files, archives, lib files and share lib files added
for an action. Its invoked twice per workflow at the moment, but the second
call does make any difference.
{code:java}
public void setLibFilesArchives(Context context, Element actionXml, Path
appPath, Configuration conf)
throws ActionExecutorException {
addWfApplicationLibs(appPath, conf, context.getProtoActionConf());
addActionXmlFilesAndArchives(actionXml, appPath, conf);
initShareLibExcluder(conf, context);
addActionLibs(appPath, conf);
addAllShareLibs(appPath, conf, context, actionXml);
}{code}
First setLibFilesArchives is invoked in JavaActionExecutor#submitLauncher and
then submitLauncher calls createLauncherConf, where its invoked again, but one
of them is unnecessary.
I tested around a bit and its probably safe to remove the setLibFilesArchives
call for example from submitLauncher.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016113#comment-17016113
]
Mate Juhasz commented on OOZIE-3574:
pinging [~zuston] :)
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016038#comment-17016038
]
Mate Juhasz commented on OOZIE-3575:
Prerequisite oozie-site config changes:
{code:java}
'oozie.service.HadoopAccessorService.supported.filesystems': '*'
'oozie.service.HadoopAccessorService.nameNode.whitelist': ''
{code}
Added s3a filesystem credentials to a simple distcp workflow:
{code:xml}
filesystem.path
${my_s3a_bucket}
${resourceManager}
${nameNode}
-update
-skipcrccheck
${basePath}/${inputDir}/data.txt
${basePath}/${outputDir}/data.txt
{code}
The following scenarios were tested:
* workflow.xml: s3a, basePath: s3a
* workflow.xml: s3a, basePath: HDFS
* workflow.xml: HDFS, basePath: s3a - extra filesystem credentials only needed
in this case
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3575-v2.patch, OOZIE-3575-v3.patch,
> OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014301#comment-17014301
]
Mate Juhasz commented on OOZIE-3575:
Thanks [~asalamon74] and [~gezapeti]! I dont like this test failure either,
trying to address it with a new patch. Still running some tests for each
scenarios, I will update with the results.
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3575-v2.patch, OOZIE-3575-v3.patch,
> OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3575:
---
Attachment: OOZIE-3575-v3.patch
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3575-v2.patch, OOZIE-3575-v3.patch,
> OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3575:
---
Attachment: OOZIE-3575-v2.patch
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3575-v2.patch, OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3575:
---
Attachment: OOZIE-3575.patch
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3575.patch
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000820#comment-17000820
]
Mate Juhasz commented on OOZIE-3575:
The recently opened OOZIE-3574 is actually a bug, but its strongly connecting
to OOZIE-3575
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3575:
---
Issue Type: Improvement (was: Bug)
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000817#comment-17000817
]
Mate Juhasz commented on OOZIE-3575:
cc: [~gezapeti] [~asalamon74] [~dionusos]
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3575) Add credential support for cloud file systems
[
https://issues.apache.org/jira/browse/OOZIE-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3575:
---
Summary: Add credential support for cloud file systems (was: Add
credential support for cloud file storages)
> Add credential support for cloud file systems
> -
>
> Key: OOZIE-3575
> URL: https://issues.apache.org/jira/browse/OOZIE-3575
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.0
> Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
>
> Oozie by default gathers delegation tokens for the nodes defined in
> _mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_
> in case of distcp actions) and for the workflow path.
> Though this implementation is good for hdfs, we dont support occasions where
> the job related resources, which we want to access in runtime are present on
> different file systems/buckets etc...
> The HDFSCredentials class should be revised to handle getting tokens for
> different cloud storages.
> *The following scenarios should be addressed:*
> Oozie should obtain delegation token in case
> * the defaultFs is cloud
> * the workload.xml is in cloud
> * input/output/auxiliary files referred from workflow are in cloud
> * (newly introduced feature) user could define filesystem credentials for the
> workflow (as its done with hive/hcat etc..) -> this would allow the user to
> handle the situation where Oozie could not decide which tokens are needed at
> launch time by default and could also get tokens for different cloud storages
> and buckets as well
> Example for credentials addition:
> {noformat}
>
>
> filesystem
> s3a://qe-s3-bucket-mst
>
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (OOZIE-3575) Add credential support for cloud file storages
Mate Juhasz created OOZIE-3575:
--
Summary: Add credential support for cloud file storages
Key: OOZIE-3575
URL: https://issues.apache.org/jira/browse/OOZIE-3575
Project: Oozie
Issue Type: Bug
Components: core
Affects Versions: 5.2.0
Reporter: Mate Juhasz
Assignee: Mate Juhasz
Fix For: trunk
Oozie by default gathers delegation tokens for the nodes defined in
_mapreduce.job.hdfs-servers_ (or _oozie.launcher.mapreduce.job.hdfs-servers_ in
case of distcp actions) and for the workflow path.
Though this implementation is good for hdfs, we dont support occasions where
the job related resources, which we want to access in runtime are present on
different file systems/buckets etc...
The HDFSCredentials class should be revised to handle getting tokens for
different cloud storages.
*The following scenarios should be addressed:*
Oozie should obtain delegation token in case
* the defaultFs is cloud
* the workload.xml is in cloud
* input/output/auxiliary files referred from workflow are in cloud
* (newly introduced feature) user could define filesystem credentials for the
workflow (as its done with hive/hcat etc..) -> this would allow the user to
handle the situation where Oozie could not decide which tokens are needed at
launch time by default and could also get tokens for different cloud storages
and buckets as well
Example for credentials addition:
{noformat}
filesystem
s3a://qe-s3-bucket-mst
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3574) JavaAction create uncorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000791#comment-17000791
]
Mate Juhasz commented on OOZIE-3574:
Thats a great finding [~zuston]! That FileSystem instance should be created
definitely with actionLibsPath instead of appPath.
Its probably a minor change in the below code part, but I did not check in
depths.
{code:java}
FileSystem fs =
Services.get().get(HadoopAccessorService.class).createFileSystem(user,
appPath.toUri(), conf);
{code}
This could be a further enhancement for making Oozie more cloud file system
friendly.
> JavaAction create uncorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-2966) OozieCLI and TestOozieCLI test class are in different packages
[ https://issues.apache.org/jira/browse/OOZIE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991422#comment-16991422 ] Mate Juhasz commented on OOZIE-2966: The idea is good and valid I think, but the solution is not that trivial unfortunately. If we move the test classes to the client project we would need to reference for example to core and fluent-job sub-projects and that would mean circular dependencies among core and client. The only solution for resolving the circularity is to restructure a lot of code, but this way I dont think it worth the time. > OozieCLI and TestOozieCLI test class are in different packages > -- > > Key: OOZIE-2966 > URL: https://issues.apache.org/jira/browse/OOZIE-2966 > Project: Oozie > Issue Type: Task > Components: tests >Affects Versions: 4.3.0 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Trivial > > OozieCLI class is in client/org/apache/oozie/cli whereas the associated test > class resides in core/org/apache/oozie/client shouldn't they belong to same > project? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3562) Fix Hive example
[
https://issues.apache.org/jira/browse/OOZIE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984312#comment-16984312
]
Mate Juhasz commented on OOZIE-3562:
[~kmarton] could you share some info about the env where it is failing? I could
see this with Java 11, so I suspected Java 8+ is not yet supported by the Hive
version we use upstream
> Fix Hive example
>
>
> Key: OOZIE-3562
> URL: https://issues.apache.org/jira/browse/OOZIE-3562
> Project: Oozie
> Issue Type: Bug
> Components: examples
>Affects Versions: 5.1.0
>Reporter: Kinga Marton
>Priority: Major
> Attachments: hive1StachTrace.txt
>
>
> Hive example in pseudo distributed mode is failing with the following error
> message:
> {code:java}
> ACTION[008-191121145732587-oozie-mart-W@hive-node] Launcher exception:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> {code}
> See the full stacktrace attached.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3549:
---
Attachment: OOZIE-3549-v4.patch
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549-v2.patch, OOZIE-3549-v3.patch,
> OOZIE-3549-v4.patch, OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983340#comment-16983340
]
Mate Juhasz commented on OOZIE-3549:
Uploaded new patch, where I modified the teardown of the failing unit tests
where we wanted to verify the call of setKeyManagerPassword method
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549-v2.patch, OOZIE-3549-v3.patch,
> OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3549:
---
Attachment: OOZIE-3549-v3.patch
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549-v2.patch, OOZIE-3549-v3.patch,
> OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3549:
---
Attachment: OOZIE-3549-v2.patch
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549-v2.patch, OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3549:
---
Attachment: (was: OOZIE-3549-v2.patch)
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549-v2.patch, OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3549:
---
Attachment: OOZIE-3549-v2.patch
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549-v2.patch, OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982579#comment-16982579
]
Mate Juhasz commented on OOZIE-3549:
Uploading a new patch, tested in a deployment. Please see the diff in
SSLServerConnectorFactory#setKeystorePass.
Some explanation on the relation of KeyManagerPassword and KeyStorePassword
from the eclipse docs:
https://www.eclipse.org/jetty/documentation/9.3.27.v20190418/configuring-ssl.html#configuring-sslcontextfactory
I would say that using a PKCS12 keystore requires the
sslContextFactory.KeyStorePassword to be set instead (or besides) of
KeyManagerPassword. If jetty's KeyManagerPassword is not set the
KeyStorePassword will be used for that as well.
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3491) Confusing System ID error message
[
https://issues.apache.org/jira/browse/OOZIE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982497#comment-16982497
]
Mate Juhasz commented on OOZIE-3491:
Oops, fixed :) thanks
> Confusing System ID error message
> -
>
> Key: OOZIE-3491
> URL: https://issues.apache.org/jira/browse/OOZIE-3491
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Labels: newbie
> Attachments: OOZIE-3491-V2.patch, OOZIE-3491-V3.patch,
> OOZIE-3491.patch
>
>
> If the System ID is longer than 10 characters Oozie trims it and prints out
> an error message:
> {noformat}
> System ID [oozie-oozi] exceeds maximum length [10], trimming
> {noformat}
> Which is confusing, because "oozie-oozi" does not exceeds 10 characters. This
> is not the original long system ID but the trimmed short one.
> I think we should fix the error message and print out both the original and
> the new value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3491) Confusing System ID error message
[
https://issues.apache.org/jira/browse/OOZIE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3491:
---
Attachment: OOZIE-3491-V3.patch
> Confusing System ID error message
> -
>
> Key: OOZIE-3491
> URL: https://issues.apache.org/jira/browse/OOZIE-3491
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Labels: newbie
> Attachments: OOZIE-3491-V2.patch, OOZIE-3491-V3.patch,
> OOZIE-3491.patch
>
>
> If the System ID is longer than 10 characters Oozie trims it and prints out
> an error message:
> {noformat}
> System ID [oozie-oozi] exceeds maximum length [10], trimming
> {noformat}
> Which is confusing, because "oozie-oozi" does not exceeds 10 characters. This
> is not the original long system ID but the trimmed short one.
> I think we should fix the error message and print out both the original and
> the new value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3491) Confusing System ID error message
[
https://issues.apache.org/jira/browse/OOZIE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3491:
---
Attachment: OOZIE-3491-V2.patch
> Confusing System ID error message
> -
>
> Key: OOZIE-3491
> URL: https://issues.apache.org/jira/browse/OOZIE-3491
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Labels: newbie
> Attachments: OOZIE-3491-V2.patch, OOZIE-3491.patch
>
>
> If the System ID is longer than 10 characters Oozie trims it and prints out
> an error message:
> {noformat}
> System ID [oozie-oozi] exceeds maximum length [10], trimming
> {noformat}
> Which is confusing, because "oozie-oozi" does not exceeds 10 characters. This
> is not the original long system ID but the trimmed short one.
> I think we should fix the error message and print out both the original and
> the new value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959011#comment-16959011
]
Mate Juhasz commented on OOZIE-3549:
Thanks [~asalamon74] for the summary. However it turned out, that the issue
might be somewhere else... there is this function
SSLServerConnectorFactory#setKeystorePass where we call
sslContextFactory.KeyManagerPassword(keystorePass) instead of
setKeyStorePassword.
The patch is working if we change this part, which I could not understand well,
so I would like to test it out better in a deployment.
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3487) Confusing E0820 error message
[
https://issues.apache.org/jira/browse/OOZIE-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3487:
---
Attachment: OOZIE-3487.patch
> Confusing E0820 error message
> -
>
> Key: OOZIE-3487
> URL: https://issues.apache.org/jira/browse/OOZIE-3487
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Attachments: OOZIE-3487.patch
>
>
> E0820 error messages are confusing:
> {noformat}
> Action user retry max [3] is over system defined max [3], re-assign to use
> system max.{noformat}
> It's not true that 3 is over 3.
>
> It's a bug in
> [LiteWorkflowStoreService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java]
> {noformat}
> if (ret > max) {
> ret = max;
> log.warn(ErrorCode.E0820.getTemplate(), ret, max);
> }
> {noformat}
> We overwrite the max value before we print out the error messge.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3487) Confusing E0820 error message
[
https://issues.apache.org/jira/browse/OOZIE-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3487:
--
Assignee: Mate Juhasz
> Confusing E0820 error message
> -
>
> Key: OOZIE-3487
> URL: https://issues.apache.org/jira/browse/OOZIE-3487
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Attachments: OOZIE-3487.patch
>
>
> E0820 error messages are confusing:
> {noformat}
> Action user retry max [3] is over system defined max [3], re-assign to use
> system max.{noformat}
> It's not true that 3 is over 3.
>
> It's a bug in
> [LiteWorkflowStoreService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java]
> {noformat}
> if (ret > max) {
> ret = max;
> log.warn(ErrorCode.E0820.getTemplate(), ret, max);
> }
> {noformat}
> We overwrite the max value before we print out the error messge.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3491) Confusing System ID error message
[
https://issues.apache.org/jira/browse/OOZIE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3491:
---
Attachment: OOZIE-3491.patch
> Confusing System ID error message
> -
>
> Key: OOZIE-3491
> URL: https://issues.apache.org/jira/browse/OOZIE-3491
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Labels: newbie
> Attachments: OOZIE-3491.patch
>
>
> If the System ID is longer than 10 characters Oozie trims it and prints out
> an error message:
> {noformat}
> System ID [oozie-oozi] exceeds maximum length [10], trimming
> {noformat}
> Which is confusing, because "oozie-oozi" does not exceeds 10 characters. This
> is not the original long system ID but the trimmed short one.
> I think we should fix the error message and print out both the original and
> the new value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3491) Confusing System ID error message
[
https://issues.apache.org/jira/browse/OOZIE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3491:
--
Assignee: Mate Juhasz
> Confusing System ID error message
> -
>
> Key: OOZIE-3491
> URL: https://issues.apache.org/jira/browse/OOZIE-3491
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Labels: newbie
>
> If the System ID is longer than 10 characters Oozie trims it and prints out
> an error message:
> {noformat}
> System ID [oozie-oozi] exceeds maximum length [10], trimming
> {noformat}
> Which is confusing, because "oozie-oozi" does not exceeds 10 characters. This
> is not the original long system ID but the trimmed short one.
> I think we should fix the error message and print out both the original and
> the new value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952869#comment-16952869
]
Mate Juhasz commented on OOZIE-3549:
Uploaded a patch, but it is still to be tested on a real deployment
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3549:
---
Attachment: OOZIE-3549.patch
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3549.patch
>
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3549) Add back support for truststore passwords
[
https://issues.apache.org/jira/browse/OOZIE-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3549:
--
Assignee: Mate Juhasz
> Add back support for truststore passwords
> -
>
> Key: OOZIE-3549
> URL: https://issues.apache.org/jira/browse/OOZIE-3549
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
>
> OOZIE-3157 removed {{oozie.https.truststore.pass}} property, because we
> (Oozie + Jetty) don't write the truststore and the password is not required
> for reading.
> This is no longer true, Java 11's keytool now defaults to creating PKCS12
> keystores instead of JKS, and according to
> [this|https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1771363]
> bug description "A JKS keystore can be read without supplying a password (or
> by supplying an empty one) while a PKCS12 keystore requires a password to be
> set."
> We should reintroduce this property and allow the it again to specify this
> password and pass it to jetty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-1974) SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in [PREP] stage
[
https://issues.apache.org/jira/browse/OOZIE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948605#comment-16948605
]
Mate Juhasz commented on OOZIE-1974:
Uploaded a patch with initial ideas... not sure about how much business value
is added by this "feature" though. Ssh unit tests are still not working, we
should fix them before, so no new tests this time
> SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in
> [PREP] stage
> --
>
> Key: OOZIE-1974
> URL: https://issues.apache.org/jira/browse/OOZIE-1974
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: trunk
>Reporter: Michalis Kongtongk
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-1974.patch
>
>
> example WF that will fail:
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Workaround is to execute the compound command in subshell eg: $(cmd1 && cmd2)
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> $(kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
> )
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Stack trace "org.apache.oozie.command.CommandException: E0800: Action it is
> not running its in [PREP] state,"
> {code}
> 2014-08-05 23:29:49,721 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() begins
> 2014-08-05 23:29:49,723 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] Attempting to copy ssh base
> scripts to remote host [m...@192-168-88-213.lunix.lan]
> 2014-08-05 23:29:52,691 INFO org.apache.oozie.servlet.CallbackServlet:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] callback for action
> [008-140805224842389-oozie-oozi-W@Ssh]
> 2014-08-05 23:29:52,714 ERROR
> org.apache.oozie.command.wf.CompletedActionXCommand:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] XException,
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:52,714 WARN
> org.apache.oozie.service.CallableQueueService$CallableWrapper:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] exception callable [callback], E0800: Action it is not running its
> in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.
[jira] [Updated] (OOZIE-1974) SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in [PREP] stage
[
https://issues.apache.org/jira/browse/OOZIE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-1974:
---
Attachment: OOZIE-1974.patch
> SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in
> [PREP] stage
> --
>
> Key: OOZIE-1974
> URL: https://issues.apache.org/jira/browse/OOZIE-1974
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: trunk
>Reporter: Michalis Kongtongk
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-1974.patch
>
>
> example WF that will fail:
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Workaround is to execute the compound command in subshell eg: $(cmd1 && cmd2)
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> $(kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
> )
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Stack trace "org.apache.oozie.command.CommandException: E0800: Action it is
> not running its in [PREP] state,"
> {code}
> 2014-08-05 23:29:49,721 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() begins
> 2014-08-05 23:29:49,723 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] Attempting to copy ssh base
> scripts to remote host [m...@192-168-88-213.lunix.lan]
> 2014-08-05 23:29:52,691 INFO org.apache.oozie.servlet.CallbackServlet:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] callback for action
> [008-140805224842389-oozie-oozi-W@Ssh]
> 2014-08-05 23:29:52,714 ERROR
> org.apache.oozie.command.wf.CompletedActionXCommand:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] XException,
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:52,714 WARN
> org.apache.oozie.service.CallableQueueService$CallableWrapper:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] exception callable [callback], E0800: Action it is not running its
> in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:57,262 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() ends
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (OOZIE-3363) Hadoop's cleanup of local directory in uber mode causing failures for sqoop actions
[
https://issues.apache.org/jira/browse/OOZIE-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reopened OOZIE-3363:
Reopening, I closed it by mistake... :)
> Hadoop's cleanup of local directory in uber mode causing failures for sqoop
> actions
> ---
>
> Key: OOZIE-3363
> URL: https://issues.apache.org/jira/browse/OOZIE-3363
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 4.3.0, 5.0.0
>Reporter: Clemens Valiente
>Priority: Blocker
>
> See related task OOZIE-2536 and discussion
> https://issues.apache.org/jira/browse/OOZIE-2536?focusedCommentId=16617206=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16617206
> the same issue cleaning up local directories also affects sqoop jobs that
> create the sqoop.xml file inbetween.
> {code:java}
> 2018-09-15 00:31:09,416 [AsyncDispatcher event handler] INFO
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl - Num completed Tasks: 1
> 2018-09-15 00:31:09,417 [AsyncDispatcher event handler] INFO
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl -
> job_1535972033593_259806Job Transitioned from RUNNING to COMMITTING
> 2018-09-15 00:31:09,419 [CommitterEvent Processor #1] INFO
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler - Processing
> the event EventType: JOB_COMMIT
> 2018-09-15 00:31:09,455 [uber-SubtaskRunner] WARN
> org.apache.hadoop.mapred.LocalContainerLauncher - Unable to delete
> unexpected local file/dir .action.xml.crc: insufficient permissions?
> 2018-09-15 00:31:09,455 [uber-SubtaskRunner] WARN
> org.apache.hadoop.mapred.LocalContainerLauncher - Unable to delete
> unexpected local file/dir .action.xml.crc: insufficient permissions?
> 2018-09-15 00:31:09,459 [CommitterEvent Processor #1] FATAL
> org.apache.hadoop.conf.Configuration - error parsing conf sqoop-site.xml
> java.io.FileNotFoundException:
> /appdata/hdfs/v7/yarn/nm/usercache/SEM/appcache/application_1535972033593_259806/container_e100_1535972033593_259806_01_01/sqoop-site.xml
> (No such file or directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:146)
> at java.io.FileInputStream.(FileInputStream.java:101)
> at
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
> at
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2483)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2554)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2507)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2413)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:984)
> at
> org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1034)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1254)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:804)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.touchz(CommitterEventHandler.java:268)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:282)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-1974) SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in [PREP] stage
[
https://issues.apache.org/jira/browse/OOZIE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938413#comment-16938413
]
Mate Juhasz commented on OOZIE-1974:
Not related, the ssh-wrapper.sh code is "designed" in a way that it does not
support compound commands
> SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in
> [PREP] stage
> --
>
> Key: OOZIE-1974
> URL: https://issues.apache.org/jira/browse/OOZIE-1974
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: trunk
>Reporter: Michalis Kongtongk
>Assignee: Mate Juhasz
>Priority: Major
>
> example WF that will fail:
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Workaround is to execute the compound command in subshell eg: $(cmd1 && cmd2)
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> $(kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
> )
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Stack trace "org.apache.oozie.command.CommandException: E0800: Action it is
> not running its in [PREP] state,"
> {code}
> 2014-08-05 23:29:49,721 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() begins
> 2014-08-05 23:29:49,723 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] Attempting to copy ssh base
> scripts to remote host [m...@192-168-88-213.lunix.lan]
> 2014-08-05 23:29:52,691 INFO org.apache.oozie.servlet.CallbackServlet:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] callback for action
> [008-140805224842389-oozie-oozi-W@Ssh]
> 2014-08-05 23:29:52,714 ERROR
> org.apache.oozie.command.wf.CompletedActionXCommand:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] XException,
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:52,714 WARN
> org.apache.oozie.service.CallableQueueService$CallableWrapper:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] exception callable [callback], E0800: Action it is not running its
> in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:57,262 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() ends
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3544) Upgrade commons-beanutils to 1.9.4
[
https://issues.apache.org/jira/browse/OOZIE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3544:
---
Attachment: OOZIE-3544.patch
> Upgrade commons-beanutils to 1.9.4
> --
>
> Key: OOZIE-3544
> URL: https://issues.apache.org/jira/browse/OOZIE-3544
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3544.patch
>
>
> Oozie does not define the {{commons-beanutils}} version, so we end up having
> multiple versions. In my last compilation we have 1.7.0, 1.9.1, and 1.9.3 at
> the same time.
> Let's consolidate it and use the latest 1.9.4 everywhere.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3544) Upgrade commons-beanutils to 1.9.4
[
https://issues.apache.org/jira/browse/OOZIE-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3544:
--
Assignee: Mate Juhasz
> Upgrade commons-beanutils to 1.9.4
> --
>
> Key: OOZIE-3544
> URL: https://issues.apache.org/jira/browse/OOZIE-3544
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: 5.2.0
>
>
> Oozie does not define the {{commons-beanutils}} version, so we end up having
> multiple versions. In my last compilation we have 1.7.0, 1.9.1, and 1.9.3 at
> the same time.
> Let's consolidate it and use the latest 1.9.4 everywhere.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-1974) SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in [PREP] stage
[
https://issues.apache.org/jira/browse/OOZIE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-1974:
--
Assignee: Mate Juhasz
> SSH Action doesn't handle compound commands eg: cmd1 && cmd2 and stuck in
> [PREP] stage
> --
>
> Key: OOZIE-1974
> URL: https://issues.apache.org/jira/browse/OOZIE-1974
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: trunk
>Reporter: Michalis Kongtongk
>Assignee: Mate Juhasz
>Priority: Major
>
> example WF that will fail:
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Workaround is to execute the compound command in subshell eg: $(cmd1 && cmd2)
> {code}
>
>
>
>
> oozie-u...@somedomain.com
> $(kinit
> oozie-u...@somedomain.com
> -k
> -t
> /home/oozie-user/oozie.keytab
>
> hdfs
> dfs
> -put
> /tmp/random-file.txt
> /tmp/random-file.txt
> )
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {code}
> Stack trace "org.apache.oozie.command.CommandException: E0800: Action it is
> not running its in [PREP] state,"
> {code}
> 2014-08-05 23:29:49,721 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() begins
> 2014-08-05 23:29:49,723 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] Attempting to copy ssh base
> scripts to remote host [m...@192-168-88-213.lunix.lan]
> 2014-08-05 23:29:52,691 INFO org.apache.oozie.servlet.CallbackServlet:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] callback for action
> [008-140805224842389-oozie-oozi-W@Ssh]
> 2014-08-05 23:29:52,714 ERROR
> org.apache.oozie.command.wf.CompletedActionXCommand:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[] APP[-]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] XException,
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:52,714 WARN
> org.apache.oozie.service.CallableQueueService$CallableWrapper:
> SERVER[192-168-88-213.lunix.lan] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] exception callable [callback], E0800: Action it is not running its
> in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> org.apache.oozie.command.CommandException: E0800: Action it is not running
> its in [PREP] state, action [008-140805224842389-oozie-oozi-W@Ssh]
> at
> org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:77)
>
> at org.apache.oozie.command.XCommand.call(XCommand.java:251)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
> 2014-08-05 23:29:57,262 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[192-168-88-213.lunix.lan] USER[mko] GROUP[-] TOKEN[] APP[Ssh-copy]
> JOB[008-140805224842389-oozie-oozi-W]
> ACTION[008-140805224842389-oozie-oozi-W@Ssh] start() ends
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3392) Refactor ShellMain to use BufferDrainer class
[
https://issues.apache.org/jira/browse/OOZIE-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3392:
--
Assignee: Mate Juhasz
> Refactor ShellMain to use BufferDrainer class
> -
>
> Key: OOZIE-3392
> URL: https://issues.apache.org/jira/browse/OOZIE-3392
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Fix For: 5.3.0
>
>
> OOZIE-3382 introduced a new {{BufferDrainer}} class to drain the standard
> output and standard error stream of an ssh process. {{ShellMain}} should also
> use this class for reading the standard output and standard error of a shell
> process to avoid code duplication.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3465) Migrate from commons-codec
[
https://issues.apache.org/jira/browse/OOZIE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937669#comment-16937669
]
Mate Juhasz commented on OOZIE-3465:
You are right [~asalamon74], thanks for pointing out! I bumped up the version
to 1.10, which is the latest among the transitive libs
> Migrate from commons-codec
> --
>
> Key: OOZIE-3465
> URL: https://issues.apache.org/jira/browse/OOZIE-3465
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3465-V2.patch, OOZIE-3465.patch
>
>
> Ooize uses {{commons-codec}} 1.4 which is almost [10
> years|https://commons.apache.org/proper/commons-codec/changes-report.html]
> old. We hardly use this library I've only found a few references to the
> {{Base64}} class in Oozie. Java8 defines a
> [Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html]
> class which we could use instead.
> We should be careful to put this jar into the sharelibs if and only if some
> other library requires it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3465) Migrate from commons-codec
[
https://issues.apache.org/jira/browse/OOZIE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3465:
---
Attachment: OOZIE-3465-V2.patch
> Migrate from commons-codec
> --
>
> Key: OOZIE-3465
> URL: https://issues.apache.org/jira/browse/OOZIE-3465
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3465-V2.patch, OOZIE-3465.patch
>
>
> Ooize uses {{commons-codec}} 1.4 which is almost [10
> years|https://commons.apache.org/proper/commons-codec/changes-report.html]
> old. We hardly use this library I've only found a few references to the
> {{Base64}} class in Oozie. Java8 defines a
> [Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html]
> class which we could use instead.
> We should be careful to put this jar into the sharelibs if and only if some
> other library requires it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3465) Migrate from commons-codec
[
https://issues.apache.org/jira/browse/OOZIE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937572#comment-16937572
]
Mate Juhasz commented on OOZIE-3465:
Andras I uploaded an initial patch in which we dont use commons-codec directly
anymore, however it comes in as a transitive dependecy from hadoop-yarn-common,
hadoop-common, hadoop-auth, hadoop-hdfs, httpclient, spark-hive, hive-exec and
pig... so its a mess :) I added exclusions where we include oozie-core, because
it brings the transitive commons-codec from direct dependencies with itself. I
tried to minimise the presence of commons-codec on the sharelib this way.
What do you think?
> Migrate from commons-codec
> --
>
> Key: OOZIE-3465
> URL: https://issues.apache.org/jira/browse/OOZIE-3465
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3465.patch
>
>
> Ooize uses {{commons-codec}} 1.4 which is almost [10
> years|https://commons.apache.org/proper/commons-codec/changes-report.html]
> old. We hardly use this library I've only found a few references to the
> {{Base64}} class in Oozie. Java8 defines a
> [Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html]
> class which we could use instead.
> We should be careful to put this jar into the sharelibs if and only if some
> other library requires it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3465) Migrate from commons-codec
[
https://issues.apache.org/jira/browse/OOZIE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3465:
---
Attachment: OOZIE-3465.patch
> Migrate from commons-codec
> --
>
> Key: OOZIE-3465
> URL: https://issues.apache.org/jira/browse/OOZIE-3465
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3465.patch
>
>
> Ooize uses {{commons-codec}} 1.4 which is almost [10
> years|https://commons.apache.org/proper/commons-codec/changes-report.html]
> old. We hardly use this library I've only found a few references to the
> {{Base64}} class in Oozie. Java8 defines a
> [Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html]
> class which we could use instead.
> We should be careful to put this jar into the sharelibs if and only if some
> other library requires it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (OOZIE-3465) Migrate from commons-codec
[
https://issues.apache.org/jira/browse/OOZIE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3465:
--
Assignee: Mate Juhasz
> Migrate from commons-codec
> --
>
> Key: OOZIE-3465
> URL: https://issues.apache.org/jira/browse/OOZIE-3465
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Major
>
> Ooize uses {{commons-codec}} 1.4 which is almost [10
> years|https://commons.apache.org/proper/commons-codec/changes-report.html]
> old. We hardly use this library I've only found a few references to the
> {{Base64}} class in Oozie. Java8 defines a
> [Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html]
> class which we could use instead.
> We should be careful to put this jar into the sharelibs if and only if some
> other library requires it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (OOZIE-3363) Hadoop's cleanup of local directory in uber mode causing failures for sqoop actions
[
https://issues.apache.org/jira/browse/OOZIE-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz resolved OOZIE-3363.
Resolution: Duplicate
> Hadoop's cleanup of local directory in uber mode causing failures for sqoop
> actions
> ---
>
> Key: OOZIE-3363
> URL: https://issues.apache.org/jira/browse/OOZIE-3363
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 4.3.0, 5.0.0
>Reporter: Clemens Valiente
>Priority: Blocker
>
> See related task OOZIE-2536 and discussion
> https://issues.apache.org/jira/browse/OOZIE-2536?focusedCommentId=16617206=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16617206
> the same issue cleaning up local directories also affects sqoop jobs that
> create the sqoop.xml file inbetween.
> {code:java}
> 2018-09-15 00:31:09,416 [AsyncDispatcher event handler] INFO
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl - Num completed Tasks: 1
> 2018-09-15 00:31:09,417 [AsyncDispatcher event handler] INFO
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl -
> job_1535972033593_259806Job Transitioned from RUNNING to COMMITTING
> 2018-09-15 00:31:09,419 [CommitterEvent Processor #1] INFO
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler - Processing
> the event EventType: JOB_COMMIT
> 2018-09-15 00:31:09,455 [uber-SubtaskRunner] WARN
> org.apache.hadoop.mapred.LocalContainerLauncher - Unable to delete
> unexpected local file/dir .action.xml.crc: insufficient permissions?
> 2018-09-15 00:31:09,455 [uber-SubtaskRunner] WARN
> org.apache.hadoop.mapred.LocalContainerLauncher - Unable to delete
> unexpected local file/dir .action.xml.crc: insufficient permissions?
> 2018-09-15 00:31:09,459 [CommitterEvent Processor #1] FATAL
> org.apache.hadoop.conf.Configuration - error parsing conf sqoop-site.xml
> java.io.FileNotFoundException:
> /appdata/hdfs/v7/yarn/nm/usercache/SEM/appcache/application_1535972033593_259806/container_e100_1535972033593_259806_01_01/sqoop-site.xml
> (No such file or directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:146)
> at java.io.FileInputStream.(FileInputStream.java:101)
> at
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
> at
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2483)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2554)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2507)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2413)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:984)
> at
> org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1034)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1254)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:804)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.touchz(CommitterEventHandler.java:268)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:282)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-2576) Oozie ssh action Cannot run program "scp"
[
https://issues.apache.org/jira/browse/OOZIE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936818#comment-16936818
]
Mate Juhasz commented on OOZIE-2576:
Its a bit late to answer, but I think the error is because adding multiple
commands are not supported in case of ssh actions.
It should work if you specify the command like below:
{noformat}
/tmp/test.sh
{noformat}
Or:
{noformat}
sh
/tmp/test.sh
{noformat}
> Oozie ssh action Cannot run program "scp"
> -
>
> Key: OOZIE-2576
> URL: https://issues.apache.org/jira/browse/OOZIE-2576
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.0.0
> Environment: Oozie server build version: 4.0.0-cdh5.2.0
>Reporter: ywheel
>Priority: Blocker
>
> We used oozie ssh action in a production environment, the following xml codes
> is the example:
> {noformat}
>
>
>
>
> root@192.168.1.154
> sh /tmp/test.sh
>
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
> However, we meet the errors below:
> {noformat}
> 2016-06-12 22:30:54,713 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Attempting to copy
> ssh base scripts to remote host [root@192.168.1.154]
> 2016-06-12 22:30:54,869 WARN org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Error while
> executing ssh EXECUTION
> 2016-06-12 22:30:54,870 WARN org.apache.oozie.command.wf.ActionStartXCommand:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Error starting
> action [ShellAction]. ErrorType [ERROR], ErrorCode [UNKOWN_ERROR], Message
> [UNKOWN_ERROR: Cannot run program "scp": error=2, No such file or directory]
> org.apache.oozie.action.ActionExecutorException: UNKOWN_ERROR: Cannot run
> program "scp": error=2, No such file or directory
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:599)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:204)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
> at org.apache.oozie.command.XCommand.call(XCommand.java:281)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.IOException: Cannot run program "scp": error=2, No such
> file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:485)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:332)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:376)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:204)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:548)
> ... 10 more
> Caused by: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.(UNIXProcess.java:135)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
> ... 17 more
> {noformat}
> I checked the codes in {{org.apa
[jira] [Updated] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3543: --- Attachment: OOZIE-3543.patch > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-3543.patch > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz reassigned OOZIE-3543: -- Assignee: Mate Juhasz > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3405:
---
Attachment: OOZIE-3405-V2.patch
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.1.0
>Reporter: Peter Orova
> Assignee: Mate Juhasz
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-3405-V1.patch, OOZIE-3405-V2.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
Re: Review Request 71425: OOZIE-3405 - SSH action shows empty error Message and Error code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71425/ --- (Updated Sept. 4, 2019, 9:10 a.m.) Review request for oozie, Andras Salamon, Denes Bodo, and Kinga Marton. Changes --- Fixed issues, added assertion message Bugs: OOZIE-3405 https://issues.apache.org/jira/browse/OOZIE-3405 Repository: oozie-git Description --- Currently, when an SSH action fails the only message that is returned is the Status. Neither the error Message nor Error code fields are filled. This makes reporting on the causes of SSH Action failures via Oozie highly impractical: the only meaningful bit of information there is on a failed SSH Action is the Status. The Status is filled based on the presence (or lack of) the .error file that is produced in case the user submitted script returns with any other value than 0. SshActionExecutor#getActionStatus ... String outFile = getRemoteFileName(context, action, "error", false, true); String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + outFile; int retVal = getReturnValue(checkErrorCmd); ... User requirement is to provide some more detailed information on the success/failure of the user-submitted script. That could be at a minimum the return value, optionally the last ~1K of the stderr that is drained. This information could then be communicated via errorMessage and ErrorCode Diffs (updated) - core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 6956cba7b core/src/main/resources/ssh-wrapper.sh e2e6f7f9b core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java d68aed094 Diff: https://reviews.apache.org/r/71425/diff/3/ Changes: https://reviews.apache.org/r/71425/diff/2-3/ Testing --- Unit testing is a bit awkward in thise case (I would like to ask for some advice), although I have managed to test on a live cluster. Thanks, Mate Juhasz
Re: Review Request 71425: OOZIE-3405 - SSH action shows empty error Message and Error code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71425/ --- (Updated Sept. 4, 2019, 9:05 a.m.) Review request for oozie, Andras Salamon, Denes Bodo, and Kinga Marton. Bugs: OOZIE-3405 https://issues.apache.org/jira/browse/OOZIE-3405 Repository: oozie-git Description --- Currently, when an SSH action fails the only message that is returned is the Status. Neither the error Message nor Error code fields are filled. This makes reporting on the causes of SSH Action failures via Oozie highly impractical: the only meaningful bit of information there is on a failed SSH Action is the Status. The Status is filled based on the presence (or lack of) the .error file that is produced in case the user submitted script returns with any other value than 0. SshActionExecutor#getActionStatus ... String outFile = getRemoteFileName(context, action, "error", false, true); String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + outFile; int retVal = getReturnValue(checkErrorCmd); ... User requirement is to provide some more detailed information on the success/failure of the user-submitted script. That could be at a minimum the return value, optionally the last ~1K of the stderr that is drained. This information could then be communicated via errorMessage and ErrorCode Diffs (updated) - core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 6956cba7b core/src/main/resources/ssh-wrapper.sh e2e6f7f9b core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java d68aed094 Diff: https://reviews.apache.org/r/71425/diff/2/ Changes: https://reviews.apache.org/r/71425/diff/1-2/ Testing --- Unit testing is a bit awkward in thise case (I would like to ask for some advice), although I have managed to test on a live cluster. Thanks, Mate Juhasz
Review Request 71425: OOZIE-3405 - SSH action shows empty error Message and Error code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71425/ --- Review request for oozie, Andras Salamon, Denes Bodo, and Kinga Marton. Bugs: OOZIE-3405 https://issues.apache.org/jira/browse/OOZIE-3405 Repository: oozie-git Description --- Currently, when an SSH action fails the only message that is returned is the Status. Neither the error Message nor Error code fields are filled. This makes reporting on the causes of SSH Action failures via Oozie highly impractical: the only meaningful bit of information there is on a failed SSH Action is the Status. The Status is filled based on the presence (or lack of) the .error file that is produced in case the user submitted script returns with any other value than 0. SshActionExecutor#getActionStatus ... String outFile = getRemoteFileName(context, action, "error", false, true); String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + outFile; int retVal = getReturnValue(checkErrorCmd); ... User requirement is to provide some more detailed information on the success/failure of the user-submitted script. That could be at a minimum the return value, optionally the last ~1K of the stderr that is drained. This information could then be communicated via errorMessage and ErrorCode Diffs - core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 6956cba7b core/src/main/resources/ssh-wrapper.sh e2e6f7f9b core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java d68aed094 Diff: https://reviews.apache.org/r/71425/diff/1/ Testing --- Unit testing is a bit awkward in thise case (I would like to ask for some advice), although I have managed to test on a live cluster. Thanks, Mate Juhasz
[jira] [Updated] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3405:
---
Attachment: OOZIE-3405-V1.patch
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Reporter: Peter Orova
> Assignee: Mate Juhasz
>Priority: Minor
> Attachments: OOZIE-3405-V1.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921406#comment-16921406
]
Mate Juhasz commented on OOZIE-3405:
Taking this over [~orova] if you dont mind :)
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Reporter: Peter Orova
> Assignee: Mate Juhasz
>Priority: Minor
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3405:
--
Assignee: Mate Juhasz (was: Peter Orova)
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Reporter: Peter Orova
> Assignee: Mate Juhasz
>Priority: Minor
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3024) email action ignores value of content_type attribute when attachment attribute is set
[
https://issues.apache.org/jira/browse/OOZIE-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887091#comment-16887091
]
Mate Juhasz edited comment on OOZIE-3024 at 7/17/19 2:20 PM:
-
[~asalamon74], I added a test for emails with attachments and verified on a
cluster as well that the fix is working.
was (Author: matijhs):
[~asalamon74], I added a test for emails with attachments and verified that the
fix is working.
> email action ignores value of content_type attribute when attachment
> attribute is set
> -
>
> Key: OOZIE-3024
> URL: https://issues.apache.org/jira/browse/OOZIE-3024
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Jim Hopper
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3024-V2.patch, OOZIE-3024.patch
>
>
> The email action ignores value of content_type attribute when attachment
> attribute is set.
> This results in an email with a html body sent as a plaintext.
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java#L251
> {code}
> if(contentType.equals("text/html")) {
> MimeBodyPart bodyHtmlPart = new MimeBodyPart();
> bodyHtmlPart.setContent(body, contentType);
> multipart.addBodyPart(bodyHtmlPart);
> }
> else{
> MimeBodyPart bodyTextPart = new MimeBodyPart();
> bodyTextPart.setText(body);
> multipart.addBodyPart(bodyTextPart);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3024) email action ignores value of content_type attribute when attachment attribute is set
[
https://issues.apache.org/jira/browse/OOZIE-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887091#comment-16887091
]
Mate Juhasz commented on OOZIE-3024:
[~asalamon74], I added a test for emails with attachments and verified that the
fix is working.
> email action ignores value of content_type attribute when attachment
> attribute is set
> -
>
> Key: OOZIE-3024
> URL: https://issues.apache.org/jira/browse/OOZIE-3024
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Jim Hopper
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3024-V2.patch, OOZIE-3024.patch
>
>
> The email action ignores value of content_type attribute when attachment
> attribute is set.
> This results in an email with a html body sent as a plaintext.
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java#L251
> {code}
> if(contentType.equals("text/html")) {
> MimeBodyPart bodyHtmlPart = new MimeBodyPart();
> bodyHtmlPart.setContent(body, contentType);
> multipart.addBodyPart(bodyHtmlPart);
> }
> else{
> MimeBodyPart bodyTextPart = new MimeBodyPart();
> bodyTextPart.setText(body);
> multipart.addBodyPart(bodyTextPart);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3024) email action ignores value of content_type attribute when attachment attribute is set
[
https://issues.apache.org/jira/browse/OOZIE-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3024:
---
Attachment: OOZIE-3024-V2.patch
> email action ignores value of content_type attribute when attachment
> attribute is set
> -
>
> Key: OOZIE-3024
> URL: https://issues.apache.org/jira/browse/OOZIE-3024
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Jim Hopper
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3024-V2.patch, OOZIE-3024.patch
>
>
> The email action ignores value of content_type attribute when attachment
> attribute is set.
> This results in an email with a html body sent as a plaintext.
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java#L251
> {code}
> if(contentType.equals("text/html")) {
> MimeBodyPart bodyHtmlPart = new MimeBodyPart();
> bodyHtmlPart.setContent(body, contentType);
> multipart.addBodyPart(bodyHtmlPart);
> }
> else{
> MimeBodyPart bodyTextPart = new MimeBodyPart();
> bodyTextPart.setText(body);
> multipart.addBodyPart(bodyTextPart);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3024) email action ignores value of content_type attribute when attachment attribute is set
[
https://issues.apache.org/jira/browse/OOZIE-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885210#comment-16885210
]
Mate Juhasz commented on OOZIE-3024:
This issue has popped up recently, so I picked it up quickly. [~asalamon74]
could you take a look?
> email action ignores value of content_type attribute when attachment
> attribute is set
> -
>
> Key: OOZIE-3024
> URL: https://issues.apache.org/jira/browse/OOZIE-3024
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Jim Hopper
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3024.patch
>
>
> The email action ignores value of content_type attribute when attachment
> attribute is set.
> This results in an email with a html body sent as a plaintext.
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java#L251
> {code}
> if(contentType.equals("text/html")) {
> MimeBodyPart bodyHtmlPart = new MimeBodyPart();
> bodyHtmlPart.setContent(body, contentType);
> multipart.addBodyPart(bodyHtmlPart);
> }
> else{
> MimeBodyPart bodyTextPart = new MimeBodyPart();
> bodyTextPart.setText(body);
> multipart.addBodyPart(bodyTextPart);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3024) email action ignores value of content_type attribute when attachment attribute is set
[
https://issues.apache.org/jira/browse/OOZIE-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3024:
---
Attachment: OOZIE-3024.patch
> email action ignores value of content_type attribute when attachment
> attribute is set
> -
>
> Key: OOZIE-3024
> URL: https://issues.apache.org/jira/browse/OOZIE-3024
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Jim Hopper
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: OOZIE-3024.patch
>
>
> The email action ignores value of content_type attribute when attachment
> attribute is set.
> This results in an email with a html body sent as a plaintext.
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java#L251
> {code}
> if(contentType.equals("text/html")) {
> MimeBodyPart bodyHtmlPart = new MimeBodyPart();
> bodyHtmlPart.setContent(body, contentType);
> multipart.addBodyPart(bodyHtmlPart);
> }
> else{
> MimeBodyPart bodyTextPart = new MimeBodyPart();
> bodyTextPart.setText(body);
> multipart.addBodyPart(bodyTextPart);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3024) email action ignores value of content_type attribute when attachment attribute is set
[
https://issues.apache.org/jira/browse/OOZIE-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3024:
--
Assignee: Mate Juhasz
> email action ignores value of content_type attribute when attachment
> attribute is set
> -
>
> Key: OOZIE-3024
> URL: https://issues.apache.org/jira/browse/OOZIE-3024
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Jim Hopper
>Assignee: Mate Juhasz
>Priority: Major
>
> The email action ignores value of content_type attribute when attachment
> attribute is set.
> This results in an email with a html body sent as a plaintext.
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java#L251
> {code}
> if(contentType.equals("text/html")) {
> MimeBodyPart bodyHtmlPart = new MimeBodyPart();
> bodyHtmlPart.setContent(body, contentType);
> multipart.addBodyPart(bodyHtmlPart);
> }
> else{
> MimeBodyPart bodyTextPart = new MimeBodyPart();
> bodyTextPart.setText(body);
> multipart.addBodyPart(bodyTextPart);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
Re: Review Request 70607: OOZIE-3196 Authorization: restrict world readability by user
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70607/ --- (Updated May 22, 2019, 11:07 a.m.) Review request for oozie. Summary (updated) - OOZIE-3196 Authorization: restrict world readability by user Bugs: OOZIE-3196 https://issues.apache.org/jira/browse/OOZIE-3196 Repository: oozie-git Description --- The current authorization model does not fit the enterprise requirements as everything is readable and writable by everyone by default. Write access can be restricted using authorization but restricting read rights is only possible via Yarn ACLs and HDFS rights which still does not prevent accessing the workflow, coordinator or bundle job’s configurations for everyone. Improve authorization so it’s possible to configure read/write access for workflows, coordinators, and bundles in a more granular way. Could involve Sentry during implementation or create and design a new system that fits the needs. The main idea was to enhance the code with an AuthorizationInterface and allow the current authorization model to be extended with different implementations of this interface. Diffs - core/src/main/java/org/apache/oozie/BundleJobBean.java e3ba331d2 core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 2a936d609 core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1e55884cc core/src/main/java/org/apache/oozie/service/AuthorizationPlugin.java PRE-CREATION core/src/main/java/org/apache/oozie/service/AuthorizationService.java 70c0ed7c5 core/src/main/java/org/apache/oozie/service/JobContext.java PRE-CREATION core/src/main/java/org/apache/oozie/service/LegacyAuthorizationPlugin.java PRE-CREATION core/src/main/java/org/apache/oozie/service/OperationContext.java PRE-CREATION core/src/main/java/org/apache/oozie/service/ResourceContext.java PRE-CREATION core/src/main/java/org/apache/oozie/service/StrictAuthorizationPlugin.java PRE-CREATION core/src/main/java/org/apache/oozie/service/UserContext.java PRE-CREATION core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java 0b873bc62 core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java dad4a10b5 core/src/main/java/org/apache/oozie/servlet/BaseJobsServlet.java 95c65fe75 core/src/main/java/org/apache/oozie/servlet/SLAServlet.java 8fad98be9 core/src/test/java/org/apache/oozie/service/DummyAuthPlugin.java PRE-CREATION core/src/test/java/org/apache/oozie/service/TestAuthorizationService.java 36e480854 Diff: https://reviews.apache.org/r/70607/diff/1/ Testing --- Thanks, Mate Juhasz
Review Request 70607: Authorization: restrict world readability by user
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70607/ --- Review request for oozie. Bugs: OOZIE-3196 https://issues.apache.org/jira/browse/OOZIE-3196 Repository: oozie-git Description --- The current authorization model does not fit the enterprise requirements as everything is readable and writable by everyone by default. Write access can be restricted using authorization but restricting read rights is only possible via Yarn ACLs and HDFS rights which still does not prevent accessing the workflow, coordinator or bundle job’s configurations for everyone. Improve authorization so it’s possible to configure read/write access for workflows, coordinators, and bundles in a more granular way. Could involve Sentry during implementation or create and design a new system that fits the needs. The main idea was to enhance the code with an AuthorizationInterface and allow the current authorization model to be extended with different implementations of this interface. Diffs - core/src/main/java/org/apache/oozie/BundleJobBean.java e3ba331d2 core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 2a936d609 core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1e55884cc core/src/main/java/org/apache/oozie/service/AuthorizationPlugin.java PRE-CREATION core/src/main/java/org/apache/oozie/service/AuthorizationService.java 70c0ed7c5 core/src/main/java/org/apache/oozie/service/JobContext.java PRE-CREATION core/src/main/java/org/apache/oozie/service/LegacyAuthorizationPlugin.java PRE-CREATION core/src/main/java/org/apache/oozie/service/OperationContext.java PRE-CREATION core/src/main/java/org/apache/oozie/service/ResourceContext.java PRE-CREATION core/src/main/java/org/apache/oozie/service/StrictAuthorizationPlugin.java PRE-CREATION core/src/main/java/org/apache/oozie/service/UserContext.java PRE-CREATION core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java 0b873bc62 core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java dad4a10b5 core/src/main/java/org/apache/oozie/servlet/BaseJobsServlet.java 95c65fe75 core/src/main/java/org/apache/oozie/servlet/SLAServlet.java 8fad98be9 core/src/test/java/org/apache/oozie/service/DummyAuthPlugin.java PRE-CREATION core/src/test/java/org/apache/oozie/service/TestAuthorizationService.java 36e480854 Diff: https://reviews.apache.org/r/70607/diff/1/ Testing --- Thanks, Mate Juhasz
[jira] [Commented] (OOZIE-3196) Authorization: restrict world readability by user
[ https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812246#comment-16812246 ] Mate Juhasz commented on OOZIE-3196: [~orova] thank you for letting me to take over the remaining tasks. I would like to get introduced with the related code parts first, then I will update the jira, hopefully with a POC. > Authorization: restrict world readability by user > - > > Key: OOZIE-3196 > URL: https://issues.apache.org/jira/browse/OOZIE-3196 > Project: Oozie > Issue Type: New Feature > Components: bundle, coordinator, workflow >Affects Versions: 5.0.0b1, 5.0.0 >Reporter: Andras Piros >Assignee: Mate Juhasz >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3196.001.patch > > > The [*current authorization > model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the > enterprise requirements as everything is readable and writable by everyone by > default. > Write access can be restricted using authorization but restricting read > rights is only possible via Yarn ACLs and HDFS rights which still does not > prevent accessing the workflow, coordinator or bundle job’s configurations > for everyone. > Improve authorization so it’s possible to configure read/write access for > workflows, coordinators, and bundles in a more granular way. Could involve > Sentry during implementation or create and design a new system that fits the > needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3196) Authorization: restrict world readability by user
[ https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz reassigned OOZIE-3196: -- Assignee: Mate Juhasz (was: Peter Orova) > Authorization: restrict world readability by user > - > > Key: OOZIE-3196 > URL: https://issues.apache.org/jira/browse/OOZIE-3196 > Project: Oozie > Issue Type: New Feature > Components: bundle, coordinator, workflow >Affects Versions: 5.0.0b1, 5.0.0 >Reporter: Andras Piros >Assignee: Mate Juhasz >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3196.001.patch > > > The [*current authorization > model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the > enterprise requirements as everything is readable and writable by everyone by > default. > Write access can be restricted using authorization but restricting read > rights is only possible via Yarn ACLs and HDFS rights which still does not > prevent accessing the workflow, coordinator or bundle job’s configurations > for everyone. > Improve authorization so it’s possible to configure read/write access for > workflows, coordinators, and bundles in a more granular way. Could involve > Sentry during implementation or create and design a new system that fits the > needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3450) Investigate and clean git sharelib
[ https://issues.apache.org/jira/browse/OOZIE-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804899#comment-16804899 ] Mate Juhasz commented on OOZIE-3450: Thanks [~asalamon74] for the testing. Could you please give the new patch a try as well? Interesting that each of the sharelib projects are using the oozie-core as a provided dependency, the only difference is that while most of them just reffering to static constants in the ActionExecutor-s, sharelib-git's GitMain tries to call a method in GitActionExecutor$ActionConfVerifier. This method is the ActionConfVerifier#checkAndGetTrimmed, which only returns an action conf value, not connecting strongly anywhere, so I moved it to the GitMain class. > Investigate and clean git sharelib > -- > > Key: OOZIE-3450 > URL: https://issues.apache.org/jira/browse/OOZIE-3450 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Major > Attachments: OOZIE-3450-v2.patch, OOZIE-3540-v1.patch > > > I've checked the number of jars in the Oozie sharelibs and realized that git > sharelib contains the highest number of jars (203), it's much more than the > hive (85), pig (67). Not to mention that we have really small sharelibs like > distcp (3). > I don't really understand the reason for this, we need to check if we really > need all the jars here. The huge number of jars make it slower and it's more > likely that we get strange errors because of jar conflicts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V4.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch, OOZIE-3443-V2.patch, > OOZIE-3443-V3.patch, OOZIE-3443-V4.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803043#comment-16803043 ] Mate Juhasz commented on OOZIE-3443: I agree [~asalamon74], but in this case I would exclude the transitive joda-time from oozie-core, so we will only package them in the necessary sharelib projects. > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch, OOZIE-3443-V2.patch, > OOZIE-3443-V3.patch, OOZIE-3443-V4.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3450) Investigate and clean git sharelib
[
https://issues.apache.org/jira/browse/OOZIE-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802711#comment-16802711
]
Mate Juhasz commented on OOZIE-3450:
I have managed to reduce to number of jars on the git sharelib (9) with better
chosen dependency scopes, but unfortunately I could not find an environment to
run the examples code. [~asalamon74] could you please help to check what
happens if we keep the below jars only? I upload a patch as well, but these are
really minor changes in code.
{noformat}
find share/lib/git -name "*.jar" | sort
share/lib/git/JavaEWAH-1.1.6.jar
share/lib/git/commons-lang3-3.3.2.jar
share/lib/git/httpclient-4.3.6.jar
share/lib/git/httpcore-4.3.3.jar
share/lib/git/jsch-0.1.54.jar
share/lib/git/jzlib-1.1.1.jar
share/lib/git/oozie-sharelib-git-5.2.0-SNAPSHOT.jar
share/lib/git/org.eclipse.jgit-5.0.1.201806211838-r.jar
share/lib/git/slf4j-api-1.6.6.jar
{noformat}
> Investigate and clean git sharelib
> --
>
> Key: OOZIE-3450
> URL: https://issues.apache.org/jira/browse/OOZIE-3450
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3540-v1.patch
>
>
> I've checked the number of jars in the Oozie sharelibs and realized that git
> sharelib contains the highest number of jars (203), it's much more than the
> hive (85), pig (67). Not to mention that we have really small sharelibs like
> distcp (3).
> I don't really understand the reason for this, we need to check if we really
> need all the jars here. The huge number of jars make it slower and it's more
> likely that we get strange errors because of jar conflicts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3450) Investigate and clean git sharelib
[ https://issues.apache.org/jira/browse/OOZIE-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3450: --- Attachment: OOZIE-3540-v1.patch > Investigate and clean git sharelib > -- > > Key: OOZIE-3450 > URL: https://issues.apache.org/jira/browse/OOZIE-3450 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon >Priority: Major > Attachments: OOZIE-3540-v1.patch > > > I've checked the number of jars in the Oozie sharelibs and realized that git > sharelib contains the highest number of jars (203), it's much more than the > hive (85), pig (67). Not to mention that we have really small sharelibs like > distcp (3). > I don't really understand the reason for this, we need to check if we really > need all the jars here. The huge number of jars make it slower and it's more > likely that we get strange errors because of jar conflicts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3443) Migrate from joda time to java.time
[
https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801857#comment-16801857
]
Mate Juhasz commented on OOZIE-3443:
I agree :) I put back the dependency to the root pom.xml to force the versions
to 2.9.9, thanks [~asalamon74]!
Now the distro will look like this:
{noformat}
./embedded-oozie-server/webapp/WEB-INF/lib/joda-time-2.9.9.jar
./share/lib/pig/joda-time-2.9.9.jar
./share/lib/hcatalog/joda-time-2.9.9.jar
./share/lib/hive2/joda-time-2.9.9.jar
./share/lib/hive/joda-time-2.9.9.jar
./share/lib/spark/joda-time-2.9.9.jar
{noformat}
> Migrate from joda time to java.time
> ---
>
> Key: OOZIE-3443
> URL: https://issues.apache.org/jira/browse/OOZIE-3443
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
> Assignee: Mate Juhasz
>Priority: Minor
> Attachments: OOZIE-3443-V1.patch, OOZIE-3443-V2.patch,
> OOZIE-3443-V3.patch
>
>
> From Java SE 8 onwards, users are asked to migrate to from [joda
> time|https://www.joda.org/joda-time/] to java.time (JSR-310).
> It seems to me, that we only use it directly in one place:
> [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java]
> so it would be not too difficult to migrate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V3.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch, OOZIE-3443-V2.patch, > OOZIE-3443-V3.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V2.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch, OOZIE-3443-V2.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V1.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: (was: OOZIE-3443-V1.patch) > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3443: --- Attachment: OOZIE-3443-V1.patch > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3443) Migrate from joda time to java.time
[ https://issues.apache.org/jira/browse/OOZIE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz reassigned OOZIE-3443: -- Assignee: Mate Juhasz > Migrate from joda time to java.time > --- > > Key: OOZIE-3443 > URL: https://issues.apache.org/jira/browse/OOZIE-3443 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Andras Salamon > Assignee: Mate Juhasz >Priority: Minor > Attachments: OOZIE-3443-V1.patch > > > From Java SE 8 onwards, users are asked to migrate to from [joda > time|https://www.joda.org/joda-time/] to java.time (JSR-310). > It seems to me, that we only use it directly in one place: > [TestWorkflowActionRetryInfoXCommand.java|https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionRetryInfoXCommand.java] > so it would be not too difficult to migrate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-1624) Exclusion pattern for sharelib JARs
[ https://issues.apache.org/jira/browse/OOZIE-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-1624: --- Attachment: OOZIE-1624-V10.patch > Exclusion pattern for sharelib JARs > --- > > Key: OOZIE-1624 > URL: https://issues.apache.org/jira/browse/OOZIE-1624 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 4.3.1 >Reporter: Purshotam Shah > Assignee: Mate Juhasz >Priority: Major > Attachments: OOZIE-1624-V10.patch, OOZIE-1624-V2.patch, > OOZIE-1624-V3.patch, OOZIE-1624-V4.patch, OOZIE-1624-V5.patch, > OOZIE-1624-v1.patch > > > Sharelib may bring some jar which might conflict with user jars. > Ex. Sharelib hive has json-2..jar, where as some of the user use-case > need higher version of json jar. > He should be able to exclude sharelib json jar and bring his own version. > > oozie.action.sharelib.for.hive.exclusion > json-\*.jar|abc-*.jar > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-1624) Exclusion pattern for sharelib JARs
[ https://issues.apache.org/jira/browse/OOZIE-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-1624: --- Attachment: OOZIE-1624-V10.patch > Exclusion pattern for sharelib JARs > --- > > Key: OOZIE-1624 > URL: https://issues.apache.org/jira/browse/OOZIE-1624 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 4.3.1 >Reporter: Purshotam Shah > Assignee: Mate Juhasz >Priority: Major > Attachments: OOZIE-1624-V10.patch, OOZIE-1624-V2.patch, > OOZIE-1624-V3.patch, OOZIE-1624-V4.patch, OOZIE-1624-V5.patch, > OOZIE-1624-v1.patch > > > Sharelib may bring some jar which might conflict with user jars. > Ex. Sharelib hive has json-2..jar, where as some of the user use-case > need higher version of json jar. > He should be able to exclude sharelib json jar and bring his own version. > > oozie.action.sharelib.for.hive.exclusion > json-\*.jar|abc-*.jar > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-1624) Exclusion pattern for sharelib JARs
[ https://issues.apache.org/jira/browse/OOZIE-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786844#comment-16786844 ] Mate Juhasz commented on OOZIE-1624: Fixed the typos and line breaks and also added the warning message for using exclusions :) > Exclusion pattern for sharelib JARs > --- > > Key: OOZIE-1624 > URL: https://issues.apache.org/jira/browse/OOZIE-1624 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 4.3.1 >Reporter: Purshotam Shah > Assignee: Mate Juhasz >Priority: Major > Attachments: OOZIE-1624-V2.patch, OOZIE-1624-V3.patch, > OOZIE-1624-V4.patch, OOZIE-1624-V5.patch, OOZIE-1624-v1.patch > > > Sharelib may bring some jar which might conflict with user jars. > Ex. Sharelib hive has json-2..jar, where as some of the user use-case > need higher version of json jar. > He should be able to exclude sharelib json jar and bring his own version. > > oozie.action.sharelib.for.hive.exclusion > json-\*.jar|abc-*.jar > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 66656: Exclusion pattern for sharelib.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66656/ --- (Updated March 7, 2019, 2:40 p.m.) Review request for oozie, András Piros and Denes Bodo. Bugs: OOZIE-1624 https://issues.apache.org/jira/browse/OOZIE-1624 Repository: oozie-git Description --- OOZIE-1624 Exclusion pattern for sharelib. Diffs (updated) - core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 231b38ea core/src/main/java/org/apache/oozie/action/hadoop/ShareLibExcluder.java PRE-CREATION core/src/main/java/org/apache/oozie/service/ShareLibService.java b88dab3a core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 05511e4c core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 6383e814 core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java PRE-CREATION core/src/test/java/org/apache/oozie/action/hadoop/TestShareLibExcluder.java PRE-CREATION docs/src/site/markdown/WorkflowFunctionalSpec.md 7d6a31bf Diff: https://reviews.apache.org/r/66656/diff/11/ Changes: https://reviews.apache.org/r/66656/diff/10-11/ Testing --- Tested on a cluster Thanks, Mate Juhasz
[jira] [Commented] (OOZIE-1624) Exclusion pattern for sharelib JARs
[ https://issues.apache.org/jira/browse/OOZIE-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786802#comment-16786802 ] Mate Juhasz commented on OOZIE-1624: Thanks [~asalamon74], I forgot to add it. Attached to the review, hope it's clear and understandable. > Exclusion pattern for sharelib JARs > --- > > Key: OOZIE-1624 > URL: https://issues.apache.org/jira/browse/OOZIE-1624 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 4.3.1 >Reporter: Purshotam Shah > Assignee: Mate Juhasz >Priority: Major > Attachments: OOZIE-1624-V2.patch, OOZIE-1624-V3.patch, > OOZIE-1624-V4.patch, OOZIE-1624-V5.patch, OOZIE-1624-v1.patch > > > Sharelib may bring some jar which might conflict with user jars. > Ex. Sharelib hive has json-2..jar, where as some of the user use-case > need higher version of json jar. > He should be able to exclude sharelib json jar and bring his own version. > > oozie.action.sharelib.for.hive.exclusion > json-\*.jar|abc-*.jar > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 66656: Exclusion pattern for sharelib.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66656/ --- (Updated March 7, 2019, 2:05 p.m.) Review request for oozie, András Piros and Denes Bodo. Bugs: OOZIE-1624 https://issues.apache.org/jira/browse/OOZIE-1624 Repository: oozie-git Description --- OOZIE-1624 Exclusion pattern for sharelib. Diffs (updated) - core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 231b38ea core/src/main/java/org/apache/oozie/action/hadoop/ShareLibExcluder.java PRE-CREATION core/src/main/java/org/apache/oozie/service/ShareLibService.java b88dab3a core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 05511e4c core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 6383e814 core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java PRE-CREATION core/src/test/java/org/apache/oozie/action/hadoop/TestShareLibExcluder.java PRE-CREATION docs/src/site/markdown/WorkflowFunctionalSpec.md 7d6a31bf Diff: https://reviews.apache.org/r/66656/diff/10/ Changes: https://reviews.apache.org/r/66656/diff/9-10/ Testing --- Tested on a cluster Thanks, Mate Juhasz
Re: Review Request 66656: Exclusion pattern for sharelib.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66656/ --- (Updated March 7, 2019, 1:49 p.m.) Review request for oozie, András Piros and Denes Bodo. Changes --- Docs added Bugs: OOZIE-1624 https://issues.apache.org/jira/browse/OOZIE-1624 Repository: oozie-git Description --- OOZIE-1624 Exclusion pattern for sharelib. Diffs (updated) - core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 231b38ea core/src/main/java/org/apache/oozie/action/hadoop/ShareLibExcluder.java PRE-CREATION core/src/main/java/org/apache/oozie/service/ShareLibService.java b88dab3a core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 05511e4c core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 6383e814 core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java PRE-CREATION core/src/test/java/org/apache/oozie/action/hadoop/TestShareLibExcluder.java PRE-CREATION docs/src/site/markdown/WorkflowFunctionalSpec.md 7d6a31bf Diff: https://reviews.apache.org/r/66656/diff/9/ Changes: https://reviews.apache.org/r/66656/diff/8-9/ Testing --- Tested on a cluster Thanks, Mate Juhasz
