[jira] [Updated] (OOZIE-3217) Enable definition of admin users using oozie-site.xml
[
https://issues.apache.org/jira/browse/OOZIE-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3217:
Description:
Currently the list of admin users is defined in the {{adminusers.txt}} file
hard coded to the Oozie config dir. For a more streamlined solution, we could
define the list of admin users via {{oozie-site.xml}} by introducing the
following configuration, which receives the comma separated values of the users
that are admins.
{{oozie.service.AuthorizationService.admin.users}}
was:
Currently the list of admin users is defined in the adminusers.txt file hard
coded to the oozie config dir. For a more streamlined solution, we could define
the list of admin users via oozie-site.xml by introducing the following
configuration, which receives the comma separated values of the users that are
admins.
oozie.service.AuthorizationService.admin.users
> Enable definition of admin users using oozie-site.xml
> -
>
> Key: OOZIE-3217
> URL: https://issues.apache.org/jira/browse/OOZIE-3217
> Project: Oozie
> Issue Type: Improvement
>Reporter: Peter Orova
>Assignee: Peter Orova
>Priority: Minor
>
> Currently the list of admin users is defined in the {{adminusers.txt}} file
> hard coded to the Oozie config dir. For a more streamlined solution, we could
> define the list of admin users via {{oozie-site.xml}} by introducing the
> following configuration, which receives the comma separated values of the
> users that are admins.
> {{oozie.service.AuthorizationService.admin.users}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[
https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433564#comment-16433564
]
Andras Piros commented on OOZIE-2494:
-
[~kmarton] thanks for the heads up!
I'd suggest following approach:
* only address Cron like syntax (so no EL DST corrections in this JIRA like
{{$\{coord:hours\(n\)\}}} or {{$\{coord:minutes\(n\)\}}}). If you like you can
open another JIRA for that one. To me it seems the community is fairly
satisfied w/ the solution applied there so far
* for Cron syntax of any kind:
** for daily {{frequency}}, go with the suggestion of the JIRA description
having the same functionality as EL {{$\{coord:days\(1\)\}}}
** for hourly {{frequency}}, have the same functionality as EL
{{$\{coord:hours\(1\)\}}}
** for {{frequency}} by the minute, have the same functionality as EL
{{$\{coord:minutes\(1\)\}}}
> Cron syntax not handling DST properly
> -
>
> Key: OOZIE-2494
> URL: https://issues.apache.org/jira/browse/OOZIE-2494
> Project: Oozie
> Issue Type: Bug
> Components: coordinator
>Affects Versions: 4.2.0
>Reporter: Dennis Pallett
>Assignee: Julia Kinga Marton
>Priority: Blocker
> Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch,
> OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch,
> testActionMaterWithDST3.patch
>
>
> When specifying a coordinator frequency, you can also specify a "timezone".
> While the frequency is always calculated in UTC, the timezone’s DST rules are
> still applied. We can see this in the following two Coordinators, which ran
> across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles
> timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job
> has “America/Los_Angeles” as the timezone. Both jobs have a frequency of
> {{$\{coord:days(1)\}}}.
> {noformat}
> Job Name: el-UTC
> Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT
> {noformat}
> {noformat}
> Job Name: el-LA
> Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT
> {noformat}
> As you can see, @2’s nominal time is adjusted to an hour earlier in the
> "el-LA" job, but not in the "el-UTC" job.
> However, when running a similar set of jobs, but using cron syntax [({{10 1
> 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day
> of every month), this isn’t the case:
> {noformat}
> Job Name: cron-UTC
> Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT
> {noformat}
> {noformat}
> Job Name: cron-LA
> Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT
> {noformat}
> As you can see, @2’s nominal time are the same in both the "cron-UTC" and
> "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the
> "el-LA" job from earlier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3196) Authorization: restrict world readability by user
[ https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3196: Affects Version/s: 5.0.0 > Authorization: restrict world readability by user > - > > Key: OOZIE-3196 > URL: https://issues.apache.org/jira/browse/OOZIE-3196 > Project: Oozie > Issue Type: New Feature > Components: bundle, coordinator, workflow >Affects Versions: 5.0.0b1, 5.0.0 >Reporter: Andras Piros >Assignee: Peter Orova >Priority: Major > Fix For: 5.1.0 > > > The [*current authorization > model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the > enterprise requirements as everything is readable and writable by everyone by > default. > Write access can be restricted using authorization but restricting read > rights is only possible via Yarn ACLs and HDFS rights which still does not > prevent accessing the workflow, coordinator or bundle job’s configurations > for everyone. > Improve authorization so it’s possible to configure read/write access for > workflows, coordinators, and bundles in a more granular way. Could involve > Sentry during implementation or create and design a new system that fits the > needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3196) Authorization: restrict world readability by user
[ https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3196: Fix Version/s: 5.1.0 > Authorization: restrict world readability by user > - > > Key: OOZIE-3196 > URL: https://issues.apache.org/jira/browse/OOZIE-3196 > Project: Oozie > Issue Type: New Feature > Components: bundle, coordinator, workflow >Affects Versions: 5.0.0b1, 5.0.0 >Reporter: Andras Piros >Assignee: Peter Orova >Priority: Major > Fix For: 5.1.0 > > > The [*current authorization > model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the > enterprise requirements as everything is readable and writable by everyone by > default. > Write access can be restricted using authorization but restricting read > rights is only possible via Yarn ACLs and HDFS rights which still does not > prevent accessing the workflow, coordinator or bundle job’s configurations > for everyone. > Improve authorization so it’s possible to configure read/write access for > workflows, coordinators, and bundles in a more granular way. Could involve > Sentry during implementation or create and design a new system that fits the > needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2683) Rewrite the Oozie Web UI
[ https://issues.apache.org/jira/browse/OOZIE-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2683: Affects Version/s: 5.0.0 > Rewrite the Oozie Web UI > > > Key: OOZIE-2683 > URL: https://issues.apache.org/jira/browse/OOZIE-2683 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Major > Fix For: 5.2.0 > > > We're currently relying on a version of ExtJS that's so old, it's not linked > on their official website, and is often temporarily deleted. Looks like it's > now been deleted for good (OOZIE-2622). It also has a GPL license, so we > can't include it with Oozie, and users are forced to go and download it > themselves. And finally, it's a really outdated UI that's not very good. > We should invest in a new Web UI that's more modern and compatibly licensed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2683) Rewrite the Oozie Web UI
[ https://issues.apache.org/jira/browse/OOZIE-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2683: Fix Version/s: 5.2.0 > Rewrite the Oozie Web UI > > > Key: OOZIE-2683 > URL: https://issues.apache.org/jira/browse/OOZIE-2683 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Major > Fix For: 5.2.0 > > > We're currently relying on a version of ExtJS that's so old, it's not linked > on their official website, and is often temporarily deleted. Looks like it's > now been deleted for good (OOZIE-2622). It also has a GPL license, so we > can't include it with Oozie, and users are forced to go and download it > themselves. And finally, it's a really outdated UI that's not very good. > We should invest in a new Web UI that's more modern and compatibly licensed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3214) Allow configurable timezone for coordinators
[
https://issues.apache.org/jira/browse/OOZIE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3214:
Description:
Right now in case of coordinators only daylight saving time corrections are
applied, the processing timezone for the coordinators is always the Oozie
processing timezone.
It would be more transparent to have an option for changing the {{timezone}}
attribute itself, such as an additional attribute in the {{coordinator.xml}}
(meaning a new version of {{coordinator.xsd}}). I would make this option
switched off by default(to have the actual behavior).
was:
Right now in case of coordinators only DST corrections are applied, the
processing timezone for the coordinators is always the Oozie processing
timezone.
It would be more transparent to have an option for changing the timezone
itself, such as an additional attribute in the coordinator.xml. I would make
this option switched off by default(to have the actual behaviour).
> Allow configurable timezone for coordinators
>
>
> Key: OOZIE-3214
> URL: https://issues.apache.org/jira/browse/OOZIE-3214
> Project: Oozie
> Issue Type: Improvement
> Components: coordinator
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
>
> Right now in case of coordinators only daylight saving time corrections are
> applied, the processing timezone for the coordinators is always the Oozie
> processing timezone.
> It would be more transparent to have an option for changing the {{timezone}}
> attribute itself, such as an additional attribute in the {{coordinator.xml}}
> (meaning a new version of {{coordinator.xsd}}). I would make this option
> switched off by default(to have the actual behavior).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431872#comment-16431872
]
Andras Piros commented on OOZIE-3186:
-
Interesting idea [~dionusos]! The overall approach looks good to me.
Can you please create unit tests that cover the original and the modified
behavior, as well as put the patch to ReviewBoard? Thanks!
> Oozie is unable to use configuration linked using jceks://file/...
> --
>
> Key: OOZIE-3186
> URL: https://issues.apache.org/jira/browse/OOZIE-3186
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Fix For: trunk
>
> Attachments: OOZIE-3186-001.patch, OOZIE-3186-002.patch
>
>
> When Oozie is used with Ambari, the next configuration makes Oozie fail to
> start:
> {noformat}
>
> hadoop.security.credential.provider.path
> jceks://file/.../oozie-site.jceks
>
> {noformat}
> Value should have *localjceks://* instead of *jceks://*. But Ambari does not
> let change this value. I propose change the url when Oozie loads it.
>
> Stacktrace, when the issue occurs:
> {code:java}
> org.apache.oozie.service.ServiceException: E0103: Could not load service
> classes, Could not load password for [oozie.service.JPAService.jdbc.password]
> at org.apache.oozie.service.Services.loadServices(Services.java:309) at
> org.apache.oozie.service.Services.init(Services.java:213) at
> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
> at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
> org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
> org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
> org.apache.catalina.core.StandardService.start(StandardService.java:525) at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
> org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
> java.lang.IllegalArgumentException: Could not load password for
> [oozie.service.JPAService.jdbc.password] at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
> at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
> at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
> org.apache.oozie.service.Services.setService(Services.java:372) at
> org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
> Caused by: java.lang.reflect.InvocationTargetException at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
> ... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
> local file system is not allowed at
> org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
> at
[jira] [Updated] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3186:
Description:
When Oozie is used with Ambari, the next configuration makes Oozie fail to
start:
{noformat}
hadoop.security.credential.provider.path
jceks://file/.../oozie-site.jceks
{noformat}
Value should have *localjceks://* instead of *jceks://*. But Ambari does not
let change this value. I propose change the url when Oozie loads it.
Stacktrace, when the issue occurs:
{noformat}
org.apache.oozie.service.ServiceException: E0103: Could not load service
classes, Could not load password for [oozie.service.JPAService.jdbc.password]
at org.apache.oozie.service.Services.loadServices(Services.java:309) at
org.apache.oozie.service.Services.init(Services.java:213) at
org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780) at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
org.apache.catalina.core.StandardService.start(StandardService.java:525) at
org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
java.lang.IllegalArgumentException: Could not load password for
[oozie.service.JPAService.jdbc.password] at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
org.apache.oozie.service.Services.setService(Services.java:372) at
org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
local file system is not allowed at
org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47) at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:82)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at
[jira] [Updated] (OOZIE-3186) Oozie is unable to use configuration linked using jceks://file/...
[
https://issues.apache.org/jira/browse/OOZIE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3186:
Description:
When Oozie is used with Ambari, the next configuration makes Oozie fail to
start:
{noformat}
hadoop.security.credential.provider.path
jceks://file/.../oozie-site.jceks
{noformat}
Value should have *localjceks://* instead of *jceks://*. But Ambari does not
let change this value. I propose change the url when Oozie loads it.
Stacktrace, when the issue occurs:
{code:java}
org.apache.oozie.service.ServiceException: E0103: Could not load service
classes, Could not load password for [oozie.service.JPAService.jdbc.password]
at org.apache.oozie.service.Services.loadServices(Services.java:309) at
org.apache.oozie.service.Services.init(Services.java:213) at
org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780) at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069) at
org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061) at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at
org.apache.catalina.core.StandardService.start(StandardService.java:525) at
org.apache.catalina.core.StandardServer.start(StandardServer.java:761) at
org.apache.catalina.startup.Catalina.start(Catalina.java:595) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by:
java.lang.IllegalArgumentException: Could not load password for
[oozie.service.JPAService.jdbc.password] at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:615)
at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:602)
at org.apache.oozie.service.JPAService.init(JPAService.java:147) at
org.apache.oozie.service.Services.setServiceInternal(Services.java:386) at
org.apache.oozie.service.Services.setService(Services.java:372) at
org.apache.oozie.service.Services.loadServices(Services.java:305) ... 26 more
Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.oozie.service.ConfigurationService.getPassword(ConfigurationService.java:608)
... 31 more Caused by: java.lang.UnsupportedOperationException: Accessing
local file system is not allowed at
org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47) at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390) at
org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:82)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at
[jira] [Commented] (OOZIE-3194) Oozie should set proper permissions to sharelib after upload
[ https://issues.apache.org/jira/browse/OOZIE-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431807#comment-16431807 ] Andras Piros commented on OOZIE-3194: - Thanks for the contribution [~dionusos]! Can you please upload the latest patch to ReviewBoard, and also add a JUnit test case where the functionality the patch tries to address gets covered? Thanks! > Oozie should set proper permissions to sharelib after upload > > > Key: OOZIE-3194 > URL: https://issues.apache.org/jira/browse/OOZIE-3194 > Project: Oozie > Issue Type: Bug > Components: tools >Affects Versions: 4.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Critical > Fix For: 4.3.1 > > Attachments: OOZIE-3194-v1.patch, OOZIE-3194-v2.patch > > > When user changes HDFS umask that does not support world readability to newly > created files, upgrading sharelib causes failing Oozie jobs. > I suggest set the required permissions to sharelib after upgrading it. Files > to 544 and directories to 755. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3017) API for workflows: other action types
[
https://issues.apache.org/jira/browse/OOZIE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3017:
Fix Version/s: (was: 5.0.0)
5.1.0
> API for workflows: other action types
> -
>
> Key: OOZIE-3017
> URL: https://issues.apache.org/jira/browse/OOZIE-3017
> Project: Oozie
> Issue Type: Sub-task
> Components: client
>Reporter: Daniel Becker
>Assignee: Andras Piros
>Priority: Major
> Fix For: 5.1.0
>
> Original Estimate: 32h
> Time Spent: 16h
> Remaining Estimate: 0h
>
> Following action types have to be implemented:
> * {{DistcpAction}}
> * {{Hive2Action}}
> * {{HiveAction}}
> * {{JavaAction}}
> * {{PigAction}}
> * {{ShellAction}}
> * {{SparkAction}}
> * {{SqoopAction}}
> * {{SshAction}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3154) CLI: generate, check, submit and run workflow definitions based on API jars
[ https://issues.apache.org/jira/browse/OOZIE-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3154: Fix Version/s: (was: 5.0.0) 5.1.0 > CLI: generate, check, submit and run workflow definitions based on API jars > --- > > Key: OOZIE-3154 > URL: https://issues.apache.org/jira/browse/OOZIE-3154 > Project: Oozie > Issue Type: Sub-task > Components: client >Affects Versions: 4.3.0 >Reporter: Andras Piros >Assignee: Andras Piros >Priority: Major > Fix For: 5.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-2683) Rewrite the Oozie Web UI
[ https://issues.apache.org/jira/browse/OOZIE-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430551#comment-16430551 ] Andras Piros edited comment on OOZIE-2683 at 4/9/18 1:52 PM: - [~matijhs] [~thai.bui] nice to see you want to contribute to the rebirth of Oozie UI! Will soon create subtasks w/ a draft on how I imagine the development process and we can continue discussion, and later, development :) was (Author: andras.piros): [~matijhs] [~thai.bui] nice to see you want to contribute to the reborn of Oozie UI! Will soon create subtasks w/ a draft on how I imagine the development process and we can continue discussion, and later, development :) > Rewrite the Oozie Web UI > > > Key: OOZIE-2683 > URL: https://issues.apache.org/jira/browse/OOZIE-2683 > Project: Oozie > Issue Type: New Feature >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Major > > We're currently relying on a version of ExtJS that's so old, it's not linked > on their official website, and is often temporarily deleted. Looks like it's > now been deleted for good (OOZIE-2622). It also has a GPL license, so we > can't include it with Oozie, and users are forced to go and download it > themselves. And finally, it's a really outdated UI that's not very good. > We should invest in a new Web UI that's more modern and compatibly licensed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2683) Rewrite the Oozie Web UI
[ https://issues.apache.org/jira/browse/OOZIE-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430551#comment-16430551 ] Andras Piros commented on OOZIE-2683: - [~matijhs] [~thai.bui] nice to see you want to contribute to the reborn of Oozie UI! Will soon create subtasks w/ a draft on how I imagine the development process and we can continue discussion, and later, development :) > Rewrite the Oozie Web UI > > > Key: OOZIE-2683 > URL: https://issues.apache.org/jira/browse/OOZIE-2683 > Project: Oozie > Issue Type: New Feature >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Major > > We're currently relying on a version of ExtJS that's so old, it's not linked > on their official website, and is often temporarily deleted. Looks like it's > now been deleted for good (OOZIE-2622). It also has a GPL license, so we > can't include it with Oozie, and users are forced to go and download it > themselves. And finally, it's a really outdated UI that's not very good. > We should invest in a new Web UI that's more modern and compatibly licensed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[
https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426618#comment-16426618
]
Andras Piros commented on OOZIE-2494:
-
[~rkanter] a fair point to maintain backwards compatibility.
Do you think users who rely on current bug in their coordinators (cron syntax
not honoring DST) won't get disappointed if we just fix it?
> Cron syntax not handling DST properly
> -
>
> Key: OOZIE-2494
> URL: https://issues.apache.org/jira/browse/OOZIE-2494
> Project: Oozie
> Issue Type: Bug
> Components: coordinator
>Affects Versions: 4.2.0
>Reporter: Dennis Pallett
>Assignee: Julia Kinga Marton
>Priority: Blocker
> Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch,
> OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch,
> testActionMaterWithDST3.patch
>
>
> When specifying a coordinator frequency, you can also specify a "timezone".
> While the frequency is always calculated in UTC, the timezone’s DST rules are
> still applied. We can see this in the following two Coordinators, which ran
> across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles
> timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job
> has “America/Los_Angeles” as the timezone. Both jobs have a frequency of
> {{$\{coord:days(1)\}}}.
> {noformat}
> Job Name: el-UTC
> Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT
> {noformat}
> {noformat}
> Job Name: el-LA
> Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT
> {noformat}
> As you can see, @2’s nominal time is adjusted to an hour earlier in the
> "el-LA" job, but not in the "el-UTC" job.
> However, when running a similar set of jobs, but using cron syntax [({{10 1
> 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day
> of every month), this isn’t the case:
> {noformat}
> Job Name: cron-UTC
> Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT
> {noformat}
> {noformat}
> Job Name: cron-LA
> Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST
> -
> ID Nominal Time (UTC) Nominal Time (LA)
> @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST
> @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT
> {noformat}
> As you can see, @2’s nominal time are the same in both the "cron-UTC" and
> "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the
> "el-LA" job from earlier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426600#comment-16426600
]
Andras Piros commented on OOZIE-3134:
-
Thanks [~kmarton] for the contribution! +1
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch, OOZIE-3134-007.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3196) Authorization: restrict world readability by user
[
https://issues.apache.org/jira/browse/OOZIE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425269#comment-16425269
]
Andras Piros commented on OOZIE-3196:
-
[~orova] just having talked to [~dbist13] offline. We think following is
important to reach a minimum viable product grade:
# be able to login / authenticate per user (configurable via {{oozie-site.xml}}
deploy time)
# most UI tabs only viewable to {{admin}}
# secure REST calls for not logged in / not {{admin}} users outside of
workflow, coordinator, and bundle listing / info
# filter workflow, coordinator, and bundle listing / info REST calls for not
logged in / not {{admin}} users
# test UI and CLI that these work like expected
> Authorization: restrict world readability by user
> -
>
> Key: OOZIE-3196
> URL: https://issues.apache.org/jira/browse/OOZIE-3196
> Project: Oozie
> Issue Type: New Feature
> Components: bundle, coordinator, workflow
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Peter Orova
>Priority: Major
>
> The [*current authorization
> model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the
> enterprise requirements as everything is readable and writable by everyone by
> default.
> Write access can be restricted using authorization but restricting read
> rights is only possible via Yarn ACLs and HDFS rights which still does not
> prevent accessing the workflow, coordinator or bundle job’s configurations
> for everyone.
> Improve authorization so it’s possible to configure read/write access for
> workflows, coordinators, and bundles in a more granular way. Could involve
> Sentry during implementation or create and design a new system that fits the
> needs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3209) XML schema error when submitting pyspark example
[
https://issues.apache.org/jira/browse/OOZIE-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424000#comment-16424000
]
Andras Piros commented on OOZIE-3209:
-
Failing tests are unrelated. Committed to {{master}}.
> XML schema error when submitting pyspark example
>
>
> Key: OOZIE-3209
> URL: https://issues.apache.org/jira/browse/OOZIE-3209
> Project: Oozie
> Issue Type: Bug
> Components: examples
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Minor
> Fix For: 5.1.0
>
> Attachments: OOZIE-3209-001.patch
>
>
> The following exception is thrown when the pyspark example is submitted:
> {code:java}
> Error: E0701 : E0701: XML schema error, cvc-complex-type.2.4.a: Invalid
> content was found starting with element 'resource-manager'. One of
> '{"uri:oozie:spark-action:0.1":job-tracker}' is expected.{code}
> The xmlns version should be changed to 1.0 in order to accept the
> resource-manager element as well.
> [https://github.com/apache/oozie/blob/master/examples/src/main/apps/3/workflow.xml#L24|https://github.com/apache/oozie/blob/master/examples/src/main/apps/pyspark/workflow.xml#L24]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3209) XML schema error when submitting pyspark example
[
https://issues.apache.org/jira/browse/OOZIE-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423974#comment-16423974
]
Andras Piros commented on OOZIE-3209:
-
Thanks for the contribution [~kmarton]! +1 (pending Jenkins)
> XML schema error when submitting pyspark example
>
>
> Key: OOZIE-3209
> URL: https://issues.apache.org/jira/browse/OOZIE-3209
> Project: Oozie
> Issue Type: Bug
> Components: examples
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-3209-001.patch
>
>
> The following exception is thrown when the pyspark example is submitted:
> {code:java}
> Error: E0701 : E0701: XML schema error, cvc-complex-type.2.4.a: Invalid
> content was found starting with element 'resource-manager'. One of
> '{"uri:oozie:spark-action:0.1":job-tracker}' is expected.{code}
> The xmlns version should be changed to 1.0 in order to accept the
> resource-manager element as well.
> [https://github.com/apache/oozie/blob/master/examples/src/main/apps/3/workflow.xml#L24|https://github.com/apache/oozie/blob/master/examples/src/main/apps/pyspark/workflow.xml#L24]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2937) Remove redundant groupId from the child POMs
[ https://issues.apache.org/jira/browse/OOZIE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2937: Summary: Remove redundant groupId from the child POMs (was: Remove redundant groupId from the child pom's) > Remove redundant groupId from the child POMs > > > Key: OOZIE-2937 > URL: https://issues.apache.org/jira/browse/OOZIE-2937 > Project: Oozie > Issue Type: Task > Components: build >Reporter: Jan Hentschel >Assignee: Jan Hentschel >Priority: Trivial > Attachments: OOZIE-2937-1.patch, OOZIE-2937-2.patch > > > Currently all child pom's define the *groupId* again. That's not necessary, > because it is already defined in the parent pom. Remove the redundant > declarations of the *groupId*. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2947) Support job-xml or global configs for credentials provider
[
https://issues.apache.org/jira/browse/OOZIE-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2947:
Description: Today, one needs to specify their HBase properties in their
credentials section by
[hand|https://github.com/apache/oozie/blob/99a3df568d182e75751126aac5fe8b63d876e64f/client/src/main/resources/oozie-workflow-0.5.xsd#L321-L341]
(or EL Expression) if acquiring an HBase credential for example. This is
despite having the same settings in {{hbase-site.xml}} e.g. for a Java action.
It would be nice if the same configurations the actions used for HBase could be
used by the credentials provider.
> Support job-xml or global configs for credentials provider
> --
>
> Key: OOZIE-2947
> URL: https://issues.apache.org/jira/browse/OOZIE-2947
> Project: Oozie
> Issue Type: New Feature
> Components: security
>Affects Versions: 4.2.0
>Reporter: Clay B.
>Priority: Minor
>
> Today, one needs to specify their HBase properties in their credentials
> section by
> [hand|https://github.com/apache/oozie/blob/99a3df568d182e75751126aac5fe8b63d876e64f/client/src/main/resources/oozie-workflow-0.5.xsd#L321-L341]
> (or EL Expression) if acquiring an HBase credential for example. This is
> despite having the same settings in {{hbase-site.xml}} e.g. for a Java
> action. It would be nice if the same configurations the actions used for
> HBase could be used by the credentials provider.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2947) Support job-xml or global configs for credentials provider
[ https://issues.apache.org/jira/browse/OOZIE-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2947: Environment: (was: Today, one needs to specify their HBase properties in their credentials section by [hand|https://github.com/apache/oozie/blob/99a3df568d182e75751126aac5fe8b63d876e64f/client/src/main/resources/oozie-workflow-0.5.xsd#L321-L341] (or EL Expression) if acquiring an HBase credential for example. This is despite having the same settings in `hbase-site.xml` e.g. for a Java action. It would be nice if the same configurations the actions used for HBase could be used by the credentials provider.) > Support job-xml or global configs for credentials provider > -- > > Key: OOZIE-2947 > URL: https://issues.apache.org/jira/browse/OOZIE-2947 > Project: Oozie > Issue Type: New Feature > Components: security >Affects Versions: 4.2.0 >Reporter: Clay B. >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2947) Support job-xml or global configs for credentials provider
[ https://issues.apache.org/jira/browse/OOZIE-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2947: Environment: Today, one needs to specify their HBase properties in their credentials section by [hand|https://github.com/apache/oozie/blob/99a3df568d182e75751126aac5fe8b63d876e64f/client/src/main/resources/oozie-workflow-0.5.xsd#L321-L341] (or EL Expression) if acquiring an HBase credential for example. This is despite having the same settings in `hbase-site.xml` e.g. for a Java action. It would be nice if the same configurations the actions used for HBase could be used by the credentials provider. (was: Today, one needs to specify their HBase properties in their credentials section by [hand|https://github.com/apache/oozie/blob/99a3df568d182e75751126aac5fe8b63d876e64f/client/src/main/resources/oozie-workflow-0.5.xsd#L321-L341[ (or EL Expression) if acquiring an HBase credential for example. This is despite having the same settings in `hbase-site.xml` e.g. for a Java action. It would be nice if the same configurations the actions used for HBase could be used by the credentials provider.) > Support job-xml or global configs for credentials provider > -- > > Key: OOZIE-2947 > URL: https://issues.apache.org/jira/browse/OOZIE-2947 > Project: Oozie > Issue Type: New Feature > Components: security >Affects Versions: 4.2.0 > Environment: Today, one needs to specify their HBase properties in > their credentials section by > [hand|https://github.com/apache/oozie/blob/99a3df568d182e75751126aac5fe8b63d876e64f/client/src/main/resources/oozie-workflow-0.5.xsd#L321-L341] > (or EL Expression) if acquiring an HBase credential for example. This is > despite having the same settings in `hbase-site.xml` e.g. for a Java action. > It would be nice if the same configurations the actions used for HBase could > be used by the credentials provider. >Reporter: Clay B. >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3199) Let system property restriction configurable
[ https://issues.apache.org/jira/browse/OOZIE-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415709#comment-16415709 ] Andras Piros commented on OOZIE-3199: - [~dionusos] thanks for the contribution! Leaving my overall opinion on ReviewBoard for the WIP patch. > Let system property restriction configurable > > > Key: OOZIE-3199 > URL: https://issues.apache.org/jira/browse/OOZIE-3199 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3199-001.patch, OOZIE-3199-002.patch > > > Currently Oozie restricts resolution of system properties by default. In some > cases that would be nice if as an administrator I could set a whitelist of > enabled system property names. > Suggestion will be provided asap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3176) Oozie-core fails with checkstyle errors
[ https://issues.apache.org/jira/browse/OOZIE-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3176: Summary: Oozie-core fails with checkstyle errors (was: Oozie-core fails with checkstyle errors.) > Oozie-core fails with checkstyle errors > --- > > Key: OOZIE-3176 > URL: https://issues.apache.org/jira/browse/OOZIE-3176 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0b1 > Environment: # uname -a > Linux in-ibmibm283.persistent.co.in 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Aug > 25 11:21:22 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Alisha Prabhu >Assignee: Alisha Prabhu >Priority: Blocker > Fix For: 5.0.0 > > Attachments: Checkstyle-erros.txt, OOZIE-3176-001.patch, > OOZIE-3176-002.patch > > > There are multiple checkstyle violations in the Oozie-core module. > Maven command used is mvn clean install -DskipTests > I have attached the Checkstyle-errors.txt file, that has the errors occurred > while building. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3176) Oozie-core fails with checkstyle errors.
[ https://issues.apache.org/jira/browse/OOZIE-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415342#comment-16415342 ] Andras Piros commented on OOZIE-3176: - Thanks for the contribution [~alishap]! +1 > Oozie-core fails with checkstyle errors. > > > Key: OOZIE-3176 > URL: https://issues.apache.org/jira/browse/OOZIE-3176 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0b1 > Environment: # uname -a > Linux in-ibmibm283.persistent.co.in 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Aug > 25 11:21:22 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Alisha Prabhu >Assignee: Alisha Prabhu >Priority: Blocker > Fix For: 5.0.0 > > Attachments: Checkstyle-erros.txt, OOZIE-3176-001.patch, > OOZIE-3176-002.patch > > > There are multiple checkstyle violations in the Oozie-core module. > Maven command used is mvn clean install -DskipTests > I have attached the Checkstyle-errors.txt file, that has the errors occurred > while building. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3150) Remove references to not present dependencies within NOTICE.txt
[
https://issues.apache.org/jira/browse/OOZIE-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3150:
Summary: Remove references to not present dependencies within NOTICE.txt
(was: Update NOTICE.txt)
> Remove references to not present dependencies within NOTICE.txt
> ---
>
> Key: OOZIE-3150
> URL: https://issues.apache.org/jira/browse/OOZIE-3150
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0b1
>Reporter: Robert Kanter
>Assignee: Peter Cseh
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-3150.001.patch
>
>
> NOTICE.txt currently contains:
> {noformat}
> ===
> NOTICE file for use with, and corresponding to Section 4 of,
> the Apache License, Version 2.0,
> in this case for the Oozie, Workflow Engine for Hadoop project
> ===
> Copyright 2011 The Apache Software Foundation
> This product includes software developed by The Apache Software
> Foundation (http://www.apache.org/).
> This product includes org.json (http://www.json.org/java/index.html),
> Copyright (c) 2002 JSON.org
> Detailed License information for all components can be found in the
> documentation in the ooziedocs.war at index.html##LicenseInfo
> Oozie source contains 3 binaries (wordcount-simple_Linux-amd64-64_h20,
> wordcount-simple_Linux-i386-32_h20, wordcount-simple_Mac_OS_X-x86_64-64_h20)
> for
> testing purposes, these binaries are compiled artifacts from Apache Hadoop.
> {noformat}
> A few problems:
> - There is no ooziedocs.war
> - Oozie doesn't contain any binaries AFAIK and hasn't for a long time
> - We should see if there is anything else we need to add
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3150) Remove references to not present dependencies within NOTICE.txt
[
https://issues.apache.org/jira/browse/OOZIE-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414037#comment-16414037
]
Andras Piros commented on OOZIE-3150:
-
Committed to {{master}}.
> Remove references to not present dependencies within NOTICE.txt
> ---
>
> Key: OOZIE-3150
> URL: https://issues.apache.org/jira/browse/OOZIE-3150
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0b1
>Reporter: Robert Kanter
>Assignee: Peter Cseh
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-3150.001.patch
>
>
> NOTICE.txt currently contains:
> {noformat}
> ===
> NOTICE file for use with, and corresponding to Section 4 of,
> the Apache License, Version 2.0,
> in this case for the Oozie, Workflow Engine for Hadoop project
> ===
> Copyright 2011 The Apache Software Foundation
> This product includes software developed by The Apache Software
> Foundation (http://www.apache.org/).
> This product includes org.json (http://www.json.org/java/index.html),
> Copyright (c) 2002 JSON.org
> Detailed License information for all components can be found in the
> documentation in the ooziedocs.war at index.html##LicenseInfo
> Oozie source contains 3 binaries (wordcount-simple_Linux-amd64-64_h20,
> wordcount-simple_Linux-i386-32_h20, wordcount-simple_Mac_OS_X-x86_64-64_h20)
> for
> testing purposes, these binaries are compiled artifacts from Apache Hadoop.
> {noformat}
> A few problems:
> - There is no ooziedocs.war
> - Oozie doesn't contain any binaries AFAIK and hasn't for a long time
> - We should see if there is anything else we need to add
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2645) Deprecate Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2645:
Attachment: OOZIE-2645.004.patch
> Deprecate Instrumentation in favor of Metrics
> -
>
> Key: OOZIE-2645
> URL: https://issues.apache.org/jira/browse/OOZIE-2645
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2645.001.patch, OOZIE-2645.002.patch,
> OOZIE-2645.003.patch, OOZIE-2645.004.patch
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5, we should drop Instrumentation and only have
> Metrics.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
>
> Update: for 5.0.0 we only deprecate {{InstrumentationService}}, and make
> {{MetricsInstrumentationService}} the default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3198) Extend remaining job tracker references in code and documentation
[
https://issues.apache.org/jira/browse/OOZIE-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3198:
Fix Version/s: (was: 5.0.0)
5.1.0
> Extend remaining job tracker references in code and documentation
> -
>
> Key: OOZIE-3198
> URL: https://issues.apache.org/jira/browse/OOZIE-3198
> Project: Oozie
> Issue Type: Sub-task
> Components: action, core, docs
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Fix For: 5.1.0
>
>
> Extend the places in configuration, code and documentation where
> {{jobTracker}} and {{job-tracker}} is still referenced without a proper
> {{resourceManager}}, {{resource-manager}} reference that precedes it.
> Example: {{oozie.service.HadoopAccessorService.jobTracker.whitelist}} in
> {{AG_HadoopConfiguration.twiki}}, {{HadoopAccessorService.java}}, and
> {{oozie-default.xml}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-3150) Update NOTICE.txt
[
https://issues.apache.org/jira/browse/OOZIE-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414009#comment-16414009
]
Andras Piros edited comment on OOZIE-3150 at 3/26/18 3:34 PM:
--
Thanks for the contribution [~gezapeti]! I'm OK to remove the conflicting
parts, and have follow-up JIRAs to clarify / generate license and dependency
reports post {{5.0.0}} timeline.
+1
was (Author: andras.piros):
Thanks for the contribution [~gezapeti]! I'm OK to remove the conflicting
parts, and have follow-up JIRAs to clarify / generate license and dependency
reports post {{5.0.0}} timeline.
+1 (pending Jenkins)
> Update NOTICE.txt
> -
>
> Key: OOZIE-3150
> URL: https://issues.apache.org/jira/browse/OOZIE-3150
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0b1
>Reporter: Robert Kanter
>Assignee: Peter Cseh
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-3150.001.patch
>
>
> NOTICE.txt currently contains:
> {noformat}
> ===
> NOTICE file for use with, and corresponding to Section 4 of,
> the Apache License, Version 2.0,
> in this case for the Oozie, Workflow Engine for Hadoop project
> ===
> Copyright 2011 The Apache Software Foundation
> This product includes software developed by The Apache Software
> Foundation (http://www.apache.org/).
> This product includes org.json (http://www.json.org/java/index.html),
> Copyright (c) 2002 JSON.org
> Detailed License information for all components can be found in the
> documentation in the ooziedocs.war at index.html##LicenseInfo
> Oozie source contains 3 binaries (wordcount-simple_Linux-amd64-64_h20,
> wordcount-simple_Linux-i386-32_h20, wordcount-simple_Mac_OS_X-x86_64-64_h20)
> for
> testing purposes, these binaries are compiled artifacts from Apache Hadoop.
> {noformat}
> A few problems:
> - There is no ooziedocs.war
> - Oozie doesn't contain any binaries AFAIK and hasn't for a long time
> - We should see if there is anything else we need to add
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3150) Update NOTICE.txt
[
https://issues.apache.org/jira/browse/OOZIE-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414009#comment-16414009
]
Andras Piros commented on OOZIE-3150:
-
Thanks for the contribution [~gezapeti]! I'm OK to remove the conflicting
parts, and have follow-up JIRAs to clarify / generate license and dependency
reports post {{5.0.0}} timeline.
+1 (pending Jenkins)
> Update NOTICE.txt
> -
>
> Key: OOZIE-3150
> URL: https://issues.apache.org/jira/browse/OOZIE-3150
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0b1
>Reporter: Robert Kanter
>Assignee: Peter Cseh
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-3150.001.patch
>
>
> NOTICE.txt currently contains:
> {noformat}
> ===
> NOTICE file for use with, and corresponding to Section 4 of,
> the Apache License, Version 2.0,
> in this case for the Oozie, Workflow Engine for Hadoop project
> ===
> Copyright 2011 The Apache Software Foundation
> This product includes software developed by The Apache Software
> Foundation (http://www.apache.org/).
> This product includes org.json (http://www.json.org/java/index.html),
> Copyright (c) 2002 JSON.org
> Detailed License information for all components can be found in the
> documentation in the ooziedocs.war at index.html##LicenseInfo
> Oozie source contains 3 binaries (wordcount-simple_Linux-amd64-64_h20,
> wordcount-simple_Linux-i386-32_h20, wordcount-simple_Mac_OS_X-x86_64-64_h20)
> for
> testing purposes, these binaries are compiled artifacts from Apache Hadoop.
> {noformat}
> A few problems:
> - There is no ooziedocs.war
> - Oozie doesn't contain any binaries AFAIK and hasn't for a long time
> - We should see if there is anything else we need to add
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2726:
Attachment: OOZIE-2726.008.patch
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch,
> OOZIE-2726.006.patch, OOZIE-2726.007.patch, OOZIE-2726.008.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3093) Test Scenarios for Oozie 5.0
[ https://issues.apache.org/jira/browse/OOZIE-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3093: Fix Version/s: (was: 5.0.0) > Test Scenarios for Oozie 5.0 > > > Key: OOZIE-3093 > URL: https://issues.apache.org/jira/browse/OOZIE-3093 > Project: Oozie > Issue Type: Test >Affects Versions: 5.0.0b1 >Reporter: Artem Ervits >Priority: Blocker > > This is umbrella jira to track test scenarios on Oozie 5.0. In my discussion > with [~gezapeti] we can have a joint effort to test different test cases. > [~clayb] please provide your input as well. > 1. Oozie HA > 2. Oozie secure cluster > 3. Run all examples > 4. Execute against every Hadoop profile > etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3199) Let system property restriction configurable
[ https://issues.apache.org/jira/browse/OOZIE-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413833#comment-16413833 ] Andras Piros commented on OOZIE-3199: - Thanks for the contribution [~dionusos]! Can you please create a ReviewBoard review request? > Let system property restriction configurable > > > Key: OOZIE-3199 > URL: https://issues.apache.org/jira/browse/OOZIE-3199 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0, 4.3.1 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3199-001.patch, OOZIE-3199-002.patch > > > Currently Oozie restricts resolution of system properties by default. In some > cases that would be nice if as an administrator I could set a whitelist of > enabled system property names. > Suggestion will be provided asap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3201) Typo in TestCoordActionInputCheckXCommand
[
https://issues.apache.org/jira/browse/OOZIE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413821#comment-16413821
]
Andras Piros commented on OOZIE-3201:
-
Committed to {{master}}.
> Typo in TestCoordActionInputCheckXCommand
> -
>
> Key: OOZIE-3201
> URL: https://issues.apache.org/jira/browse/OOZIE-3201
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.0.0, 4.3.1
>Reporter: gongchuanjie
>Assignee: gongchuanjie
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: OOZIE-3201.001.patch
>
>
> the typo of "aciton" in TestCoordActionInputCheckXCommand class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3201) Typo in TestCoordActionInputCheckXCommand
[ https://issues.apache.org/jira/browse/OOZIE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3201: Fix Version/s: 5.0.0 > Typo in TestCoordActionInputCheckXCommand > - > > Key: OOZIE-3201 > URL: https://issues.apache.org/jira/browse/OOZIE-3201 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.0.0, 4.3.1 >Reporter: gongchuanjie >Assignee: gongchuanjie >Priority: Minor > Fix For: 5.0.0 > > Attachments: OOZIE-3201.001.patch > > > the typo of "aciton" in TestCoordActionInputCheckXCommand class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3201) Typo in TestCoordActionInputCheckXCommand
[ https://issues.apache.org/jira/browse/OOZIE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3201: Summary: Typo in TestCoordActionInputCheckXCommand (was: the typo in TestCoordActionInputCheckXCommand class) > Typo in TestCoordActionInputCheckXCommand > - > > Key: OOZIE-3201 > URL: https://issues.apache.org/jira/browse/OOZIE-3201 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.0.0, 4.3.1 >Reporter: gongchuanjie >Assignee: gongchuanjie >Priority: Minor > Attachments: OOZIE-3201.001.patch > > > the typo of "aciton" in TestCoordActionInputCheckXCommand class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3201) Typo in TestCoordActionInputCheckXCommand
[ https://issues.apache.org/jira/browse/OOZIE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413805#comment-16413805 ] Andras Piros commented on OOZIE-3201: - Thanks for the contribution [~gongchuanjie]! Added you to the contributors list. +1 > Typo in TestCoordActionInputCheckXCommand > - > > Key: OOZIE-3201 > URL: https://issues.apache.org/jira/browse/OOZIE-3201 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.0.0, 4.3.1 >Reporter: gongchuanjie >Assignee: gongchuanjie >Priority: Minor > Attachments: OOZIE-3201.001.patch > > > the typo of "aciton" in TestCoordActionInputCheckXCommand class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3201) the typo in TestCoordActionInputCheckXCommand class
[ https://issues.apache.org/jira/browse/OOZIE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros reassigned OOZIE-3201: --- Assignee: gongchuanjie > the typo in TestCoordActionInputCheckXCommand class > --- > > Key: OOZIE-3201 > URL: https://issues.apache.org/jira/browse/OOZIE-3201 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.0.0, 4.3.1 >Reporter: gongchuanjie >Assignee: gongchuanjie >Priority: Minor > Attachments: OOZIE-3201.001.patch > > > the typo of "aciton" in TestCoordActionInputCheckXCommand class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3056) Implement new mechanism to specify ShareLibs for workflow actions
[
https://issues.apache.org/jira/browse/OOZIE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413778#comment-16413778
]
Andras Piros commented on OOZIE-3056:
-
Thanks for the contribution [~gezapeti]! +1
> Implement new mechanism to specify ShareLibs for workflow actions
> -
>
> Key: OOZIE-3056
> URL: https://issues.apache.org/jira/browse/OOZIE-3056
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Peter Cseh
>Priority: Critical
> Fix For: 5.0.0
>
> Attachments: OOZIE-3056.001.patch, OOZIE-3056.002.patch,
> OOZIE-3056.003.patch, OOZIE-3056.004.patch, OOZIE-3056.005.patch
>
>
> OOZIE-2687 introduces the {{launcher}} element for workflows:
> {code}
>
> 1024
> 1
> -Dsome.property=true -XX:+RandomJVMSwitch
> key=value
> root.oozie
> spark,hive
>
> {code}
> The purpose of this ticket is to discuss and implement new mechanism for
> handling ShareLib.
> {{addActionShareLib}} in {{JavaActionExecutor}} should adjusted. Regarding
> "precedence order":
> if global and an action level {{launcher}} and {{configuration}} (e.g.
> {{oozie.action.sharelib.for.#ACTIONTYPE#}}) tries to override the sharelib
> the following should apply:
> {quote}
> config properties defined in an action's have priority over
> an action's , which has priority over the global section's
> and , which has priority over the action defaults in
> oozie-site, and so on.
> {quote}
> Here we have multiple choices how to handle sharelib:
> - Alternative 1: override sharelib in a way that is consistent with current
> way of handling Oozie configuration settings.
> - Alternative 2: make sharelib additive
> -- For example, if there is a global {{launcher}} with {{sharelib}} element
> in a workflow that includes multiple ShareLibs (e.g. A,B), and
> {{oozie.action.sharelib.for.#ACTIONTYPE#}} is also specified for an action's
> configuration (e.g. C,D), then we take the union of the specified entities
> (A,B,C,D would be included).
> -- It's inconsistent with everything else
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2809) Oozie HA with secure zookeeper access fails because ZKUtils#setJaasConfiguration does not replace "_HOST" in principal string
[
https://issues.apache.org/jira/browse/OOZIE-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2809:
Fix Version/s: 5.1.0
> Oozie HA with secure zookeeper access fails because
> ZKUtils#setJaasConfiguration does not replace "_HOST" in principal string
> -
>
> Key: OOZIE-2809
> URL: https://issues.apache.org/jira/browse/OOZIE-2809
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: trunk, 4.2.0, 4.3.0
>Reporter: Thomas Larsson
>Priority: Major
> Fix For: 5.1.0
>
>
> Setting "oozie.zookeeper.secure" = true causes the oozie server not to start
> if "oozie.service.HadoopAccessorService.kerberos.principal" is defined with a
> "_HOST" token instead of an actual hostname.
> The symptom when this happens is that the oozie log shows something like this:
> {noformat}
> org.apache.oozie.service.ServiceException: E1700: Issue communicating with
> ZooKeeper: KeeperErrorCode = NoAuth for /oozie/services/servers
> at
> org.apache.oozie.service.ZKLocksService.init(ZKLocksService.java:76)
> at
> org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
> at org.apache.oozie.service.Services.setService(Services.java:372)
> at org.apache.oozie.service.Services.loadServices(Services.java:305)
> at org.apache.oozie.service.Services.init(Services.java:213)
> at
> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802)
> at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
> at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
> at
> org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:676)
> at
> org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:602)
> at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:503)
> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
> at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
> at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068)
> at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060)
> at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
> at
> org.apache.catalina.core.StandardService.start(StandardService.java:525)
> at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:759)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> Caused by: org.apache.zookeeper.KeeperException$NoAuthException:
> KeeperErrorCode = NoAuth for /oozie/services/servers
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
> at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:199)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:682)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431)
> at
>
[jira] [Commented] (OOZIE-2645) Deprecate Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412777#comment-16412777
]
Andras Piros commented on OOZIE-2645:
-
[~gezapeti] failing tests are unrelated. Can you please review? Thanks!
> Deprecate Instrumentation in favor of Metrics
> -
>
> Key: OOZIE-2645
> URL: https://issues.apache.org/jira/browse/OOZIE-2645
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2645.001.patch, OOZIE-2645.002.patch,
> OOZIE-2645.003.patch
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5, we should drop Instrumentation and only have
> Metrics.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
>
> Update: for 5.0.0 we only deprecate {{InstrumentationService}}, and make
> {{MetricsInstrumentationService}} the default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412717#comment-16412717
]
Andras Piros commented on OOZIE-2726:
-
Failing tests are unrelated. [~gezapeti] can you please review? Thanks!
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch,
> OOZIE-2726.006.patch, OOZIE-2726.007.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2645) Deprecate Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2645:
Attachment: OOZIE-2645.003.patch
> Deprecate Instrumentation in favor of Metrics
> -
>
> Key: OOZIE-2645
> URL: https://issues.apache.org/jira/browse/OOZIE-2645
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2645.001.patch, OOZIE-2645.002.patch,
> OOZIE-2645.003.patch
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5, we should drop Instrumentation and only have
> Metrics.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
>
> Update: for 5.0.0 we only deprecate {{InstrumentationService}}, and make
> {{MetricsInstrumentationService}} the default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2645) Deprecate Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2645:
Attachment: OOZIE-2645.002.patch
> Deprecate Instrumentation in favor of Metrics
> -
>
> Key: OOZIE-2645
> URL: https://issues.apache.org/jira/browse/OOZIE-2645
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2645.001.patch, OOZIE-2645.002.patch
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5, we should drop Instrumentation and only have
> Metrics.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
>
> Update: for 5.0.0 we only deprecate {{InstrumentationService}}, and make
> {{MetricsInstrumentationService}} the default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2726:
Attachment: OOZIE-2726.007.patch
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch,
> OOZIE-2726.006.patch, OOZIE-2726.007.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2967) TestStatusTransitService.testBundleStatusCoordSubmitFails fails intermittently in Apache Oozie Core 5.0.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/OOZIE-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2967:
Fix Version/s: 5.1.0
> TestStatusTransitService.testBundleStatusCoordSubmitFails fails
> intermittently in Apache Oozie Core 5.0.0-SNAPSHOT
> --
>
> Key: OOZIE-2967
> URL: https://issues.apache.org/jira/browse/OOZIE-2967
> Project: Oozie
> Issue Type: Sub-task
> Components: tests
>Affects Versions: 5.0.0
> Environment: $ uname -a
> Linux pts00450-vm24 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15
> 17:42:36 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Parita Johari
>Priority: Minor
> Fix For: 5.1.0
>
>
> {code:java}
> Stacktrace
> junit.framework.AssertionFailedError: expected: but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:253)
> at
> org.apache.oozie.service.TestStatusTransitService.testBundleStatusCoordSubmitFails(TestStatusTransitService.java:1647)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3111) [Umbrella] Fix flaky tests in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3111: Fix Version/s: (was: 5.0.0) > [Umbrella] Fix flaky tests in Oozie > --- > > Key: OOZIE-3111 > URL: https://issues.apache.org/jira/browse/OOZIE-3111 > Project: Oozie > Issue Type: Bug > Components: tests >Reporter: Peter Bacsko >Assignee: Peter Cseh >Priority: Major > > In Oozie, we've had a number or flaky tests for quite a while. In order to > have successful builds, we circumvented the problem by re-running failed > tests. But this is not a good long-term solution. > This JIRA tracks the tests that are currently flaky. We should address each > of these tests to have stable builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2968) TestJavaActionExecutor.testCredentialsSkip fails intermittently in Apache Oozie Core 5.0.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/OOZIE-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2968:
Fix Version/s: 5.1.0
> TestJavaActionExecutor.testCredentialsSkip fails intermittently in Apache
> Oozie Core 5.0.0-SNAPSHOT
> ---
>
> Key: OOZIE-2968
> URL: https://issues.apache.org/jira/browse/OOZIE-2968
> Project: Oozie
> Issue Type: Sub-task
> Components: tests
>Affects Versions: 5.0.0
> Environment: $ uname -a
> Linux pts00450-vm24 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15
> 17:42:36 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Parita Johari
>Priority: Minor
> Fix For: 5.1.0
>
>
> {code:java}
> Stacktrace
> org.apache.oozie.action.ActionExecutorException: JA020: Could not load
> credentials of type [abc] with name [abcname]]; perhaps it was not defined in
> oozie-site.xml?
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1202)
> at
> org.apache.oozie.action.hadoop.TestJavaActionExecutor._testCredentialsSkip(TestJavaActionExecutor.java:1096)
> at
> org.apache.oozie.action.hadoop.TestJavaActionExecutor.testCredentialsSkip(TestJavaActionExecutor.java:995)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3105) testJMXInstrumentation from the org.apache.oozie.util.TestMetricsInstrumentation class is flaky
[
https://issues.apache.org/jira/browse/OOZIE-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3105:
Fix Version/s: 5.1.0
> testJMXInstrumentation from the
> org.apache.oozie.util.TestMetricsInstrumentation class is flaky
> ---
>
> Key: OOZIE-3105
> URL: https://issues.apache.org/jira/browse/OOZIE-3105
> Project: Oozie
> Issue Type: Sub-task
> Components: tests
>Affects Versions: 4.3.0
> Environment: mvn: 3.5.0
> java: 1.8.0.144
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3105-00.patch
>
>
> I'm able to reproduce the test failure consistently by running test in
> isolation with the following command
> {code}
> mvn clean
> -Dtest=org.apache.oozie.util.TestMetricsInstrumentation#testJMXInstrumentation
> test
> {code}
> the test succeeds consistently with the following
> {code}
> mvn clean -Dtest=org.apache.oozie.util.TestMetricsInstrumentation test
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2726:
Fix Version/s: 5.0.0
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch,
> OOZIE-2726.006.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2459: Fix Version/s: 5.1.0 > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Sub-task > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Peter Bacsko >Priority: Minor > Fix For: 5.1.0 > > Attachments: OOZIE-2459-1.patch > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2726:
Attachment: OOZIE-2726.006.patch
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch,
> OOZIE-2726.006.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411682#comment-16411682
]
Andras Piros edited comment on OOZIE-2726 at 3/23/18 4:41 PM:
--
[~satishsaley] taking over on the base of {{OOZIE\-2726\-4.patch}}.
was (Author: andras.piros):
[~satishsaley] taking over on the base of {{OOZIE-2726-4.patch}}.
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411682#comment-16411682
]
Andras Piros commented on OOZIE-2726:
-
[~satishsaley] taking over on the base of {{OOZIE-2726-4.patch}}.
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros reassigned OOZIE-2726:
---
Assignee: Andras Piros (was: Satish Subhashrao Saley)
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Andras Piros
>Priority: Blocker
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3200) Drop Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3200:
Description:
OOZIE-1817 added the option to use DropWizard Metrics instead of our homegrown
Instrumentation. We left the Instrumentation as the default for compatibility;
in Oozie 5.1.0, we should drop Instrumentation and only have Metrics.
OOZIE-2645 deprecated {{Instrumentation}} and made {{Metrics}} the default
choice, but no code cleanup happened there.
We can also use this opportunity to clean up the code and interface for
Metrics, which currently has to conform to Instrumentation for pluggability.
> Drop Instrumentation in favor of Metrics
>
>
> Key: OOZIE-3200
> URL: https://issues.apache.org/jira/browse/OOZIE-3200
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5.1.0, we should drop Instrumentation and only have
> Metrics.
> OOZIE-2645 deprecated {{Instrumentation}} and made {{Metrics}} the default
> choice, but no code cleanup happened there.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OOZIE-3200) Drop Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros reassigned OOZIE-3200:
---
Assignee: Andras Piros
> Drop Instrumentation in favor of Metrics
>
>
> Key: OOZIE-3200
> URL: https://issues.apache.org/jira/browse/OOZIE-3200
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5.1.0, we should drop Instrumentation and only have
> Metrics.
> OOZIE-2645 deprecated {{Instrumentation}} and made {{Metrics}} the default
> choice, but no code cleanup happened there.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3200) Drop Instrumentation in favor of Metrics
Andras Piros created OOZIE-3200: --- Summary: Drop Instrumentation in favor of Metrics Key: OOZIE-3200 URL: https://issues.apache.org/jira/browse/OOZIE-3200 Project: Oozie Issue Type: Improvement Components: monitoring Affects Versions: 5.0.0 Reporter: Andras Piros Fix For: 5.1.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2645) Deprecate Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2645:
Description:
OOZIE-1817 added the option to use DropWizard Metrics instead of our homegrown
Instrumentation. We left the Instrumentation as the default for compatibility;
in Oozie 5, we should drop Instrumentation and only have Metrics.
We can also use this opportunity to clean up the code and interface for
Metrics, which currently has to conform to Instrumentation for pluggability.
Update: for 5.0.0 we only deprecate {{InstrumentationService}}, and make
{{MetricsInstrumentationService}} the default.
was:
OOZIE-1817 added the option to use DropWizard Metrics instead of our homegrown
Instrumentation. We left the Instrumentation as the default for compatibility;
in Oozie 5, we should drop Instrumentation and only have Metrics.
We can also use this opportunity to clean up the code and interface for
Metrics, which currently has to conform to Instrumentation for pluggability.
> Deprecate Instrumentation in favor of Metrics
> -
>
> Key: OOZIE-2645
> URL: https://issues.apache.org/jira/browse/OOZIE-2645
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2645.001.patch
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5, we should drop Instrumentation and only have
> Metrics.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
>
> Update: for 5.0.0 we only deprecate {{InstrumentationService}}, and make
> {{MetricsInstrumentationService}} the default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2645) Deprecate Instrumentation in favor of Metrics
[ https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2645: Summary: Deprecate Instrumentation in favor of Metrics (was: Drop Instrumentation in favor of Metrics) > Deprecate Instrumentation in favor of Metrics > - > > Key: OOZIE-2645 > URL: https://issues.apache.org/jira/browse/OOZIE-2645 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Blocker > Fix For: 5.0.0 > > Attachments: OOZIE-2645.001.patch > > > OOZIE-1817 added the option to use DropWizard Metrics instead of our > homegrown Instrumentation. We left the Instrumentation as the default for > compatibility; in Oozie 5, we should drop Instrumentation and only have > Metrics. > We can also use this opportunity to clean up the code and interface for > Metrics, which currently has to conform to Instrumentation for pluggability. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2645) Drop Instrumentation in favor of Metrics
[ https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2645: Attachment: OOZIE-2645.001.patch > Drop Instrumentation in favor of Metrics > > > Key: OOZIE-2645 > URL: https://issues.apache.org/jira/browse/OOZIE-2645 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Andras Piros >Priority: Blocker > Fix For: 5.0.0 > > Attachments: OOZIE-2645.001.patch > > > OOZIE-1817 added the option to use DropWizard Metrics instead of our > homegrown Instrumentation. We left the Instrumentation as the default for > compatibility; in Oozie 5, we should drop Instrumentation and only have > Metrics. > We can also use this opportunity to clean up the code and interface for > Metrics, which currently has to conform to Instrumentation for pluggability. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411208#comment-16411208
]
Andras Piros commented on OOZIE-3134:
-
[~kmarton] please address {{RAW_PATCH_ANALYSIS}} and {{FINDBUGS_DIFF}} errors,
warnings. Thanks!
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database
[
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411007#comment-16411007
]
Andras Piros commented on OOZIE-3134:
-
Thanks for the contribution [~kmarton]! +1 (pending Jenkins)
> Potential inconsistency between the in-memory SLA map and the Oozie database
>
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch,
> OOZIE-3134-003.patch, OOZIE-3134-004.patch, OOZIE-3134-005.patch,
> OOZIE-3134-006.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an
> in-memory concurrent hashmap ("slaMap"), and later [persists
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
> in the Oozie database.
> However, if there is a failure during the database operation, a
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the
> SLA map|
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be
> implemented. It would also make sense to do more sanity/consistency check in
> the Oozie server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2645) Drop Instrumentation in favor of Metrics
[
https://issues.apache.org/jira/browse/OOZIE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410101#comment-16410101
]
Andras Piros commented on OOZIE-2645:
-
[~rkanter] going for deprecating {{Instrumentation}} and making
{{MetricsInstrumentation}} the default, and creating a follow-up JIRA to drop
{{Instrumentation}} and clean up code later.
> Drop Instrumentation in favor of Metrics
>
>
> Key: OOZIE-2645
> URL: https://issues.apache.org/jira/browse/OOZIE-2645
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
>
> OOZIE-1817 added the option to use DropWizard Metrics instead of our
> homegrown Instrumentation. We left the Instrumentation as the default for
> compatibility; in Oozie 5, we should drop Instrumentation and only have
> Metrics.
> We can also use this opportunity to clean up the code and interface for
> Metrics, which currently has to conform to Instrumentation for pluggability.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3093) Test Scenarios for Oozie 5.0
[
https://issues.apache.org/jira/browse/OOZIE-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410097#comment-16410097
]
Andras Piros commented on OOZIE-3093:
-
Hi [~dbist13]!
We internally at Cloudera test a couple of things, including Oozie HA, secure
Oozie, against a variety of Hadoop profiles (including Hadoop 3 now), running
most of the examples just to name a few. So speaking of 5.0.0 quality, this is
something we (Cloudera) can rely on.
Though not an {{5.0.0}} blocker, I consider this umbrella JIRA very important:
* for the sake of reproducible and automatized testing scenarios
* less efforts needed as part of every minor / maintenance release cycle
Sseveral important scenarios are not present in Apache Oozie, but would be
useful to implement:
* have {{MiniOozieTestCase}} instances that cover most action types like Spark
(standalone mode only), Hive, Hive 2. It's already possible to run asynchronous
Oozie actions, see {{TestWorkflow}} for details. Best would be to use
{{MiniMRYARNCluster}}, efforts are tracked in
[*OOZIE-2729*|https://issues.apache.org/jira/browse/OOZIE-2729]
* have test cases that spin up multiple Oozie server instances. At the moment
{{LocalOozie}} spins up exactly one {{MiniOozie}} - we need to have support more
* have a kerberized {{MiniDFSCluster}} / {{MiniMRCluster}} /
{{MiniMRYARNCluster}} setup, or, at the very least, some virtualization step
where examples are submitted in a kerberized environment
[~gezapeti] [~rkanter] what are your two cents?
> Test Scenarios for Oozie 5.0
>
>
> Key: OOZIE-3093
> URL: https://issues.apache.org/jira/browse/OOZIE-3093
> Project: Oozie
> Issue Type: Test
>Affects Versions: 5.0.0b1
>Reporter: Artem Ervits
>Priority: Blocker
> Fix For: 5.0.0
>
>
> This is umbrella jira to track test scenarios on Oozie 5.0. In my discussion
> with [~gezapeti] we can have a joint effort to test different test cases.
> [~clayb] please provide your input as well.
> 1. Oozie HA
> 2. Oozie secure cluster
> 3. Run all examples
> 4. Execute against every Hadoop profile
> etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3056) Implement new mechanism to specify ShareLibs for workflow actions
[
https://issues.apache.org/jira/browse/OOZIE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409578#comment-16409578
]
Andras Piros commented on OOZIE-3056:
-
Thanks for the contribution [~gezapeti]! +1 (pending Jenkins)
> Implement new mechanism to specify ShareLibs for workflow actions
> -
>
> Key: OOZIE-3056
> URL: https://issues.apache.org/jira/browse/OOZIE-3056
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Peter Cseh
>Priority: Critical
> Fix For: 5.0.0
>
> Attachments: OOZIE-3056.001.patch, OOZIE-3056.002.patch
>
>
> OOZIE-2687 introduces the {{launcher}} element for workflows:
> {code}
>
> 1024
> 1
> -Dsome.property=true -XX:+RandomJVMSwitch
> key=value
> root.oozie
> spark,hive
>
> {code}
> The purpose of this ticket is to discuss and implement new mechanism for
> handling ShareLib.
> {{addActionShareLib}} in {{JavaActionExecutor}} should adjusted. Regarding
> "precedence order":
> if global and an action level {{launcher}} and {{configuration}} (e.g.
> {{oozie.action.sharelib.for.#ACTIONTYPE#}}) tries to override the sharelib
> the following should apply:
> {quote}
> config properties defined in an action's have priority over
> an action's , which has priority over the global section's
> and , which has priority over the action defaults in
> oozie-site, and so on.
> {quote}
> Here we have multiple choices how to handle sharelib:
> - Alternative 1: override sharelib in a way that is consistent with current
> way of handling Oozie configuration settings.
> - Alternative 2: make sharelib additive
> -- For example, if there is a global {{launcher}} with {{sharelib}} element
> in a workflow that includes multiple ShareLibs (e.g. A,B), and
> {{oozie.action.sharelib.for.#ACTIONTYPE#}} is also specified for an action's
> configuration (e.g. C,D), then we take the union of the specified entities
> (A,B,C,D would be included).
> -- It's inconsistent with everything else
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2339) Provide an API for writing jobs based on the XSD schemas
[
https://issues.apache.org/jira/browse/OOZIE-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2339:
Attachment: OOZIE-2339.011.patch
> Provide an API for writing jobs based on the XSD schemas
>
>
> Key: OOZIE-2339
> URL: https://issues.apache.org/jira/browse/OOZIE-2339
> Project: Oozie
> Issue Type: New Feature
> Components: client
>Affects Versions: 4.3.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2339.001.patch, OOZIE-2339.002.patch,
> OOZIE-2339.003.patch, OOZIE-2339.004.patch, OOZIE-2339.005.patch,
> OOZIE-2339.006.patch, OOZIE-2339.008.patch, OOZIE-2339.010.patch,
> OOZIE-2339.011.patch
>
>
> Users often complain about the XML they have to write for Oozie jobs. It
> would be nice if they could write them in something like Java, but we don't
> want to have to maintain a separate Java API for this. I was looking around
> and saw that JAXB might be the right thing here. From what I can tell, it
> lets you create Java classes from XSD schemas. So, we should be able to
> auto-generate a Java API for writing Oozie jobs, without having to really
> maintain it.
> We should investigate if this is feasible and, if so, implement it.
> Some useful looking links:
> * [JAXB
> overview|https://en.wikipedia.org/wiki/Java_Architecture_for_XML_Binding]
> * [JAXB description|https://jaxb.java.net/2.2.11/docs/ch03.html]
> * [Maven JAXB plugin|https://java.net/projects/maven-jaxb2-plugin/pages/Home]
> * [Apache Falcon|https://falcon.apache.org]
> Key features:
> * must have:
> ** inside an {{oozie-jobs-api}} artifact
> ** able to create workflow / coordinator / bundle definitions programmatically
> ** synchronizing each and every XSD change on rebuild
> ** can write {{workflow.xml}}, {{coordinator.xml}}, {{bundle.xml}}, and
> {{jobs.properties}} artifacts of every XSD version
> ** cloneability of workflow etc. {{Object}} s
> ** perform cross checks, e.g. that the workflow graph is a DAG
> ** only latest XSD versions should be supported as must have
> * nice to have:
> ** XSD version(s) can be provided. When not provided, latest ones are
> considered as valid
> ** implement a [*fluent API*|https://en.wikipedia.org/wiki/Fluent_interface]
> ** have a Python / Jython REPL to make it easy to experiment
> ** create documentation about usage
> ** can read {{workflow.xml}}, {{coordinator.xml}}, {{bundle.xml}}, and
> {{jobs.properties}} artifacts of every XSD version
> ** can convert between XSD versions
> ** support XSD change on the fly (within REPL)
> ** support HDFS reads / writes
> ** support dry run on an Oozie server to perform checks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2600:
Attachment: OOZIE-2600.004.patch
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2600.001.patch, OOZIE-2600.002.patch,
> OOZIE-2600.003.patch, OOZIE-2600.004.patch
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2600:
Attachment: OOZIE-2600.003.patch
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2600.001.patch, OOZIE-2600.002.patch,
> OOZIE-2600.003.patch
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406188#comment-16406188
]
Andras Piros commented on OOZIE-2600:
-
Addressing review comments.
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2600.001.patch, OOZIE-2600.002.patch
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2600:
Attachment: OOZIE-2600.002.patch
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2600.001.patch, OOZIE-2600.002.patch
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405286#comment-16405286
]
Andras Piros edited comment on OOZIE-2600 at 3/19/18 8:14 PM:
--
Following is covered here:
* document OOZIE-2896
* upgrade to 1.0 XSDs across documentation
* rework examples to use 1.0 XSDs across documentation
* cover {{OozieClient}} / {{MiniOozie}} extensions
[~gezapeti] [~rkanter] do you see anything more within the existing
documentation that is worth reworking / updating?
was (Author: andras.piros):
Following is covered here:
* document OOZIE-2896
* upgrade to 1.0 XSDs across documentation
* rework examples to use 1.0 XSDs across documentation
* cover {{OozieClient}} / {{MiniOozie}} extensions
[~gezapeti] do you see anything more within the existing documentation that is
worth reworking / updating?
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2600.001.patch
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2600:
Attachment: OOZIE-2600.001.patch
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2600.001.patch
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2600) OYA: Update Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405286#comment-16405286
]
Andras Piros commented on OOZIE-2600:
-
Following is covered here:
* document OOZIE-2896
* upgrade to 1.0 XSDs across documentation
* rework examples to use 1.0 XSDs across documentation
* cover {{OozieClient}} / {{MiniOozie}} extensions
[~gezapeti] do you see anything more within the existing documentation that is
worth reworking / updating?
> OYA: Update Documentation
> -
>
> Key: OOZIE-2600
> URL: https://issues.apache.org/jira/browse/OOZIE-2600
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: oya, 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
>
> Based on the work done in OOZIE-2896, document also the way how
> {{oozie.launcher.override.\*}} and {{oozie.launcher.prepend.\*}} parameters
> function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3198) Extend remaining job tracker references in code and documentation
Andras Piros created OOZIE-3198:
---
Summary: Extend remaining job tracker references in code and
documentation
Key: OOZIE-3198
URL: https://issues.apache.org/jira/browse/OOZIE-3198
Project: Oozie
Issue Type: Sub-task
Components: action, core, docs
Affects Versions: 5.0.0b1
Reporter: Andras Piros
Assignee: Andras Piros
Fix For: 5.0.0
Extend the places in configuration, code and documentation where {{jobTracker}}
and {{job-tracker}} is still referenced without a proper {{resourceManager}},
{{resource-manager}} reference that precedes it.
Example: {{oozie.service.HadoopAccessorService.jobTracker.whitelist}} in
{{AG_HadoopConfiguration.twiki}}, {{HadoopAccessorService.java}}, and
{{oozie-default.xml}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3195) Typo in WebServicesAPI.twiki: Proxy Hive Job Submission
[
https://issues.apache.org/jira/browse/OOZIE-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404553#comment-16404553
]
Andras Piros commented on OOZIE-3195:
-
Thanks for the contribution [~kmarton]! +1
> Typo in WebServicesAPI.twiki: Proxy Hive Job Submission
> ---
>
> Key: OOZIE-3195
> URL: https://issues.apache.org/jira/browse/OOZIE-3195
> Project: Oozie
> Issue Type: Bug
> Components: docs
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-3195-001.patch
>
>
> There is a typo at the Proxy Hive Job Submission description.
> [https://github.com/apache/oozie/blob/master/docs/src/site/twiki/WebServicesAPI.twiki#L851]
> * =oozie.hive.options.size=: The number of options you'll be passing to
> {color:#d04437}Pig. {color:#33}It has to be Hive, not Pig.{color}{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3197) Can you please guide me how to to exception handling at action level in oozie
[
https://issues.apache.org/jira/browse/OOZIE-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404549#comment-16404549
]
Andras Piros commented on OOZIE-3197:
-
[~kunu] please have a look at [*Decision Control
Node*|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a3.1.4_Decision_Control_Node],
and [*Error
Transition*|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a3.2.1.3_Actions_Have_2_Transitions_ok_and_error],
and [*{{wf:lastErrorNode()}}, {{wf:errorCode(String node)}} and
{{wf:errorMessage(String message)}} EL
function*|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a4.2.3_Workflow_EL_Functions]
documentation sections.
> Can you please guide me how to to exception handling at action level in oozie
> -
>
> Key: OOZIE-3197
> URL: https://issues.apache.org/jira/browse/OOZIE-3197
> Project: Oozie
> Issue Type: Bug
> Components: action
> Environment: i am using oozie 4.2 and distribution is hortonworks.
>Reporter: kunal kishore
>Priority: Blocker
>
> I am developing workflow and i would like to do exception handling at action
> level, if it is possible to implement exception handling at action level.
>
> Please guide me with example or code snippet.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3196) Authorization: restrict world readability by user
Andras Piros created OOZIE-3196: --- Summary: Authorization: restrict world readability by user Key: OOZIE-3196 URL: https://issues.apache.org/jira/browse/OOZIE-3196 Project: Oozie Issue Type: New Feature Components: bundle, coordinator, workflow Affects Versions: 5.0.0b1 Reporter: Andras Piros The [*current authorization model*|https://issues.apache.org/jira/browse/OOZIE-228] does not fit the enterprise requirements as everything is readable and writable by everyone by default. Write access can be restricted using authorization but restricting read rights is only possible via Yarn ACLs and HDFS rights which still does not prevent accessing the workflow, coordinator or bundle job’s configurations for everyone. Improve authorization so it’s possible to configure read/write access for workflows, coordinators, and bundles in a more granular way. Could involve Sentry during implementation or create and design a new system that fits the needs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2880) Improve documentation on Oozie authentication and authorization configuration
[
https://issues.apache.org/jira/browse/OOZIE-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2880:
Affects Version/s: 5.0.0b1
> Improve documentation on Oozie authentication and authorization configuration
> -
>
> Key: OOZIE-2880
> URL: https://issues.apache.org/jira/browse/OOZIE-2880
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
>Priority: Major
>
> h4. Authentication
> [Documentation of Oozie
> authentication|https://oozie.apache.org/docs/4.3.0/AG_Install.html#Oozie_User_Authentication_Configuration]
> is slightly incorrect.
> {quote}
> Pseudo/simple authentication requires the user to specify the user name on
> the request, this is done by the PseudoAuthenticator class by injecting the
> user.name parameter in the query string of all requests. The user.name
> parameter value is taken from the client process Java System property
> user.name .
> {quote}
> Actually, when someone performs an Oozie operation using the CLI, a hadoop
> auth token is created and saved to {{~/.oozie-auth-token}}. In subsequent
> actions, the token is retrieved from this cache file (until the token is
> expired). In other words, passing {{user.name}} as system property to the
> Oozie CLI in an unsecure environment (or using kinit -kt in a Kerberized
> cluster) takes no effect if {{.oozie-auth-token}} is present in the user's
> home and the token is still valid.
> With {{simple}} authentication type pseudo/simple authentication is used.
> However, in an unsecure environment I tested with hadoop 2.4 (default hadoop
> version) that
> [KerberosAuthenticator|https://github.com/apache/hadoop/blob/branch-2.4.0/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java#L188]
> returns a null token. In turn, admin operations cannot be used if there is
> no {{~/.oozie-auth-token}} with a valid token. Note: this was fixed by
> [HADOOP-11467|https://github.com/apache/hadoop/commit/875256834b892b574499d5fe68f95a9aed244f7d#diff-28df14cad207bee984f5ca4820bacabcR198].
> h4. Authorization
> Regarding [authorization configuration |
> https://oozie.apache.org/docs/4.3.0/AG_Install.html#User_Authorization_Configuration]
> the current documentation mentions a deprecated configuration property
> {{oozie.service.AuthorizationService.security.enabled}}. If a user specifies
> it, the following warning is logged in Oozie server log.
> {noformat}
> 2017-05-02 03:30:59,578 WARN org.apache.oozie.util.ConfigUtils:
> SERVER[myserver.com] Using a deprecated configuration property
> [oozie.service.Authorization
> Service.security.enabled], should use
> [oozie.service.AuthorizationService.authorization.enabled]. Please delete
> the deprecated property in order for the new property to take effect.
> {noformat}
> {{oozie.service.AuthorizationService.authorization.enabled}} should be used.
> Oozie authorization only make sense if *authentication (kerberos) is
> enabled*. Otherwise any user can claim any identity (as "simple"
> authentication type uses Pseudo authentication).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2941) Old map-reduce workflow example can't work with OYA
[ https://issues.apache.org/jira/browse/OOZIE-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396795#comment-16396795 ] Andras Piros commented on OOZIE-2941: - [~Dongying Jiao] can you please re-test the issue after installing the Oozie sharelib? Thanks! > Old map-reduce workflow example can't work with OYA > --- > > Key: OOZIE-2941 > URL: https://issues.apache.org/jira/browse/OOZIE-2941 > Project: Oozie > Issue Type: Sub-task > Components: workflow >Affects Versions: trunk >Reporter: Dongying Jiao >Priority: Major > Attachments: RM_UI.jpg > > > map-reduce workflow in Oozie examples run failed after OYA merged. The > exception in yarn log is: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/mapred/Mapper > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:763) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) > at java.net.URLClassLoader.access$100(URLClassLoader.java:73) > at java.net.URLClassLoader$1.run(URLClassLoader.java:368) > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:361) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540) > at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170) > at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148) > Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 15 more -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-2814) OYA: Update example workflows to newest schemas
[ https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2814: Fix Version/s: 5.0.0 > OYA: Update example workflows to newest schemas > --- > > Key: OOZIE-2814 > URL: https://issues.apache.org/jira/browse/OOZIE-2814 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari >Priority: Blocker > Fix For: 5.0.0 > > Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, > OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch, > OOZIE-2814-05.patch, OOZIE-2814-06.patch, OOZIE-2814-07.patch > > > OOZIE-2687 will add in a whole set of newer schemas that are more > Yarn-centric. Most, if not all, examples currently use pretty old versions > of the schemas. We should take this opportunity to update the examples to > the newest schemas added by OOZIE-2687, which should help demonstrate them. > It may make sense to keep the older schemas too, to show that older workflows > still work. We have some examples where we have two workflow XML files that > do the same thing in different ways. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3111) [Umbrella] Fix flaky tests in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3111: Fix Version/s: 5.0.0 > [Umbrella] Fix flaky tests in Oozie > --- > > Key: OOZIE-3111 > URL: https://issues.apache.org/jira/browse/OOZIE-3111 > Project: Oozie > Issue Type: Bug > Components: tests >Reporter: Peter Bacsko >Priority: Major > Fix For: 5.0.0 > > > In Oozie, we've had a number or flaky tests for quite a while. In order to > have successful builds, we circumvented the problem by re-running failed > tests. But this is not a good long-term solution. > This JIRA tracks the tests that are currently flaky. We should address each > of these tests to have stable builds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3056) Implement new mechanism to specify ShareLibs for workflow actions
[
https://issues.apache.org/jira/browse/OOZIE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3056:
Fix Version/s: 5.0.0
> Implement new mechanism to specify ShareLibs for workflow actions
> -
>
> Key: OOZIE-3056
> URL: https://issues.apache.org/jira/browse/OOZIE-3056
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Priority: Major
> Fix For: 5.0.0
>
>
> OOZIE-2687 introduces the {{launcher}} element for workflows:
> {code}
>
> 1024
> 1
> -Dsome.property=true -XX:+RandomJVMSwitch
> key=value
> root.oozie
> spark,hive
>
> {code}
> The purpose of this ticket is to discuss and implement new mechanism for
> handling ShareLib.
> {{addActionShareLib}} in {{JavaActionExecutor}} should adjusted. Regarding
> "precedence order":
> if global and an action level {{launcher}} and {{configuration}} (e.g.
> {{oozie.action.sharelib.for.#ACTIONTYPE#}}) tries to override the sharelib
> the following should apply:
> {quote}
> config properties defined in an action's have priority over
> an action's , which has priority over the global section's
> and , which has priority over the action defaults in
> oozie-site, and so on.
> {quote}
> Here we have multiple choices how to handle sharelib:
> - Alternative 1: override sharelib in a way that is consistent with current
> way of handling Oozie configuration settings.
> - Alternative 2: make sharelib additive
> -- For example, if there is a global {{launcher}} with {{sharelib}} element
> in a workflow that includes multiple ShareLibs (e.g. A,B), and
> {{oozie.action.sharelib.for.#ACTIONTYPE#}} is also specified for an action's
> configuration (e.g. C,D), then we take the union of the specified entities
> (A,B,C,D would be included).
> -- It's inconsistent with everything else
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OOZIE-3108) org.apache.oozie.tools.TestDBLoadDump#testImportInvalidDataLeavesTablesEmpty is flaky
[
https://issues.apache.org/jira/browse/OOZIE-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros resolved OOZIE-3108.
-
Resolution: Cannot Reproduce
Cannot reproduce on current {{master}}.
> org.apache.oozie.tools.TestDBLoadDump#testImportInvalidDataLeavesTablesEmpty
> is flaky
> -
>
> Key: OOZIE-3108
> URL: https://issues.apache.org/jira/browse/OOZIE-3108
> Project: Oozie
> Issue Type: Sub-task
> Components: tests
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Priority: Major
>
> {code}
> mvn clean -Dtest=org.apache.oozie.tools.TestDBLoadDump test
> {code}
> passes
> {code}
> mvn clean
> -Dtest=org.apache.oozie.tools.TestDBLoadDump#testImportInvalidDataLeavesTablesEmpty
> test
> {code}
> fails on individual run.
> {code}
> mvn clean
> -Dtest=org.apache.oozie.tools.TestDBLoadDump#testImportInvalidDataLeavesTablesEmpty
> test
> {noformat}
> Running org.apache.oozie.tools.TestDBLoadDump
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.625 sec
> <<< FAILURE!
> Results :
> Failed tests:
> testImportInvalidDataLeavesTablesEmpty(org.apache.oozie.tools.TestDBLoadDump):
> import should have been ended prematurely
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Oozie Main .. SUCCESS [ 0.604
> s]
> [INFO] Apache Oozie Client SUCCESS [ 2.634
> s]
> [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 3.449
> s]
> [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ 1.468
> s]
> [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ 0.550
> s]
> [INFO] Apache Oozie Core .. SUCCESS [ 15.741
> s]
> [INFO] Apache Oozie Share Lib Streaming ... SUCCESS [ 3.544
> s]
> [INFO] Apache Oozie Share Lib Pig . SUCCESS [ 2.476
> s]
> [INFO] Apache Oozie Share Lib Hive SUCCESS [ 2.439
> s]
> [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 3.954
> s]
> [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [ 1.552
> s]
> [INFO] Apache Oozie Examples .. SUCCESS [ 2.189
> s]
> [INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 3.243
> s]
> [INFO] Apache Oozie Share Lib . SUCCESS [ 0.010
> s]
> [INFO] Apache Oozie Docs .. SUCCESS [ 0.248
> s]
> [INFO] Apache Oozie WebApp SUCCESS [ 2.731
> s]
> [INFO] Apache Oozie Tools . FAILURE [ 26.358
> s]
> [INFO] Apache Oozie MiniOozie . SKIPPED
> [INFO] Apache Oozie Server SKIPPED
> [INFO] Apache Oozie Distro SKIPPED
> [INFO] Apache Oozie ZooKeeper Security Tests .. SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 01:13 min
> [INFO] Finished at: 2017-11-01T10:49:22-04:00
> [INFO] Final Memory: 552M/1955M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.12.2:test (default-test) on
> project oozie-tools: There are test failures.
> [ERROR]
> [ERROR] Please refer to
> /Users/aervits/NetBeansProjects/OOZIE/oozie/tools/target/surefire-reports for
> the individual test results.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :oozie-tools
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2726) Flaky test due to daylight saving changes
[
https://issues.apache.org/jira/browse/OOZIE-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396754#comment-16396754
]
Andras Piros commented on OOZIE-2726:
-
[~satishsaley] do you have a chance to have a look?
> Flaky test due to daylight saving changes
> -
>
> Key: OOZIE-2726
> URL: https://issues.apache.org/jira/browse/OOZIE-2726
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Blocker
> Attachments: OOZIE-2726-1.patch, OOZIE-2726-2.patch,
> OOZIE-2726-3.patch, OOZIE-2726-4.patch, OOZIE-2726-5.patch
>
>
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup
> {code}
> Error Message
> expected: but was:
> Stacktrace
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:74)
> at
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterizationLookup(TestCoordMaterializeTransitionXCommand.java:582)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OOZIE-3055) Release Oozie 5.0.0
[
https://issues.apache.org/jira/browse/OOZIE-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros resolved OOZIE-3055.
-
Resolution: Won't Fix
> Release Oozie 5.0.0
> ---
>
> Key: OOZIE-3055
> URL: https://issues.apache.org/jira/browse/OOZIE-3055
> Project: Oozie
> Issue Type: Task
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Priority: Major
> Fix For: 5.0.0
>
>
> It's been a while where we had a decent {{4.3.0}} release. Let's roll for
> {{5.0.0}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2975) code clean up in pig sharelib, replace Exception with more explicit, add try with resources, StringBuilder instead of StringBuffer
[ https://issues.apache.org/jira/browse/OOZIE-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-2975: Fix Version/s: (was: 5.0.0) 5.1.0 > code clean up in pig sharelib, replace Exception with more explicit, add try > with resources, StringBuilder instead of StringBuffer > -- > > Key: OOZIE-2975 > URL: https://issues.apache.org/jira/browse/OOZIE-2975 > Project: Oozie > Issue Type: Improvement >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Trivial > Labels: newbie > Fix For: 5.1.0 > > Attachments: OOZIE-2975-0.patch > > > 1. Replaced StringBuffer with StringBuilder > 2. Replaced Exception with IOException and ExecException > 3. Replaced old Hashmap notation with diamond notation > 4. Added try-with-resources notation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3061) Kill only those child jobs which are not already killed
[ https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-3061: Fix Version/s: (was: 5.0.0) 5.1.0 > Kill only those child jobs which are not already killed > --- > > Key: OOZIE-3061 > URL: https://issues.apache.org/jira/browse/OOZIE-3061 > Project: Oozie > Issue Type: Bug >Affects Versions: 4.3.0 >Reporter: Satish Subhashrao Saley >Assignee: Mate Juhasz >Priority: Trivial > Labels: newbie, newbiee > Fix For: 5.1.0 > > Attachments: OOZIE-3061-001.patch, OOZIE-3061-002.patch, > OOZIE-3061-003.patch, OOZIE-3061-004.patch > > > Here we kill all child jobs. > https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265 > We should check before killing for already killed application. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-1779) Add parent-id as a filter option
[ https://issues.apache.org/jira/browse/OOZIE-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros updated OOZIE-1779: Fix Version/s: (was: 5.0.0) 5.1.0 > Add parent-id as a filter option > > > Key: OOZIE-1779 > URL: https://issues.apache.org/jira/browse/OOZIE-1779 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Mona Chitnis >Assignee: Azrael Seoeun >Priority: Minor > Fix For: 5.1.0 > > Attachments: OOZIE-1779.1.patch, OOZIE-1779.2.patch > > > Currently filtering options on jobs list are > name, user, group, status, frequency and time-unit. > With increasing usage of coordinators and bundles, adding parent-id should be > useful instead of always searching top-down from parent to child. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3184) Test case in oozie-core fails with a NullPointerException
[
https://issues.apache.org/jira/browse/OOZIE-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394998#comment-16394998
]
Andras Piros commented on OOZIE-3184:
-
Thanks [~alishap] for your work on the issue!
> Test case in oozie-core fails with a NullPointerException
> -
>
> Key: OOZIE-3184
> URL: https://issues.apache.org/jira/browse/OOZIE-3184
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Labels: ppc64le
> Attachments: error.txt
>
>
> Maven command used is mvn -Dtest=TestGraphGenerator#testGraphWithManyNodes
> test
> Observed that the test case fails at GraphGenerator.java:87 of oozie-core
> i.e. at the .parse function.
> Have attached the error.txt file below , that has the errors occurred while
> running the test case.
> {code:java}
> [INFO] Running org.apache.oozie.util.graph.TestGraphGenerator [ERROR] Tests
> run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 24.157 s <<<
> FAILURE! - in org.apache.oozie.util.graph.TestGraphGenerator [ERROR]
> testGraphWithManyNodes(org.apache.oozie.util.graph.TestGraphGenerator) Time
> elapsed: 24.157 s <<< ERROR! java.lang.RuntimeException:
> java.util.concurrent.ExecutionException: java.lang.NullPointerException at
> org.apache.oozie.util.graph.TestGraphGenerator.testGraphWithManyNodes(TestGraphGenerator.java:158)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException at
> org.apache.oozie.util.graph.TestGraphGenerator.testGraphWithManyNodes(TestGraphGenerator.java:158)
> Caused by: java.lang.NullPointerException{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3190) Test cases in TestGraphGenerator.java of oozie-core fail with a NullPointerException
[
https://issues.apache.org/jira/browse/OOZIE-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394997#comment-16394997
]
Andras Piros commented on OOZIE-3190:
-
Thanks for your work [~alishap] on the issue!
> Test cases in TestGraphGenerator.java of oozie-core fail with a
> NullPointerException
>
>
> Key: OOZIE-3190
> URL: https://issues.apache.org/jira/browse/OOZIE-3190
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Labels: ppc64le
>
> The test cases testSimpleGraphPng and testSimpleGraphSvg under
> TestGraphGenerator.java fail with java.lang.NullPointerException.
> Maven command used for testSimpleGraphPng is mvn
> -Dtest=TestGraphGenerator#testSimpleGraphPng test.
> {code:java}
> [INFO] Running org.apache.oozie.util.graph.TestGraphGenerator
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 24.229 s <<< FAILURE! - in org.apache.oozie.util.graph.TestGraphGenerator
> [ERROR] testSimpleGraphPng(org.apache.oozie.util.graph.TestGraphGenerator)
> Time elapsed: 24.229 s <<< FAILURE!
> java.lang.AssertionError: Render and write PNG failed for
> graph-workflow-simple.xml: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> at
> org.apache.oozie.util.graph.TestGraphGenerator.generateAndAssertPng(TestGraphGenerator.java:96)
> at
> org.apache.oozie.util.graph.TestGraphGenerator.testSimpleGraphPng(TestGraphGenerator.java:64)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestGraphGenerator.testSimpleGraphPng:64->generateAndAssertPng:96
> Render and write PNG failed for graph-workflow-simple.xml:
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> {code}
> Maven command used for testSimpleGraphSvg is mvn
> -Dtest=TestGraphGenerator#testSimpleGraphSvg test.
> {code:java}
> [INFO] Running org.apache.oozie.util.graph.TestGraphGenerator
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 24.113 s <<< FAILURE! - in org.apache.oozie.util.graph.TestGraphGenerator
> [ERROR] testSimpleGraphSvg(org.apache.oozie.util.graph.TestGraphGenerator)
> Time elapsed: 24.113 s <<< FAILURE!
> java.lang.AssertionError: Render and write SVG failed:
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at
> org.apache.oozie.util.graph.TestGraphGenerator.testSimpleGraphSvg(TestGraphGenerator.java:144)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestGraphGenerator.testSimpleGraphSvg:144 Render and write SVG
> failed: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> {code}
> Have observed that the test cases use the values from the enum OutputFormat
> which contains 3 constants : PNG, DOT & SVG
> The test cases using OutputFormat.DOT have passed whereas the once which have
> used the other two constants i.e PNG and SVG , fail on ppc64le arch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3182) Oozie components fail with checkstyle errors
[
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383237#comment-16383237
]
Andras Piros commented on OOZIE-3182:
-
Thanks for the contribution [~alishap]! +1. Committed to {{master}}
> Oozie components fail with checkstyle errors
>
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
> Issue Type: Bug
> Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3182-001.patch, OOZIE-3182-004.patch,
> error-log.txt, log-OOZIE-3182-004.txt
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples,
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3187) coordinator: EL function to get coordinator information: last successful run, last run status, ...
[ https://issues.apache.org/jira/browse/OOZIE-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16382389#comment-16382389 ] Andras Piros commented on OOZIE-3187: - [~rubenvw] thanks for the idea! Can you please elaborate more on the use case where a workflow job would need information about properties of the last successful *coordinator* job details? I could see more value in knowing in a workflow job that was triggered by the coordinator the status / error message of the previous / last *workflow* job triggered by the same coordinator, as: * coordinator jobs / applications lifecycle is totally different from workflow jobs / applications lifecycle * by design, workflows shouldn't know about parent coordinators (if any), as coordinators shouldn't know about parent bundles (if any) > coordinator: EL function to get coordinator information: last successful run, > last run status, ... > -- > > Key: OOZIE-3187 > URL: https://issues.apache.org/jira/browse/OOZIE-3187 > Project: Oozie > Issue Type: New Feature > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Ruben Van Wanzeele >Priority: Major > > It would be handy if I could retrieve the last successful run of my > coordinator job and pass that as a property to my workflow. > I tried to manage this, but there is no such function. Currently, you cannot > retrieve any information about the coordinator job. I could think of other > useful info such as: > * status last run > * errorcode last run > * amount of running instances > * ... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3185) Conflicting JARs org.apache.derby exist in Oozie
[
https://issues.apache.org/jira/browse/OOZIE-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16382372#comment-16382372
]
Andras Piros commented on OOZIE-3185:
-
Thanks for the contribution [~PandaMonkey]! Can you please upload a patch, and
set the issue to {{Patch Available}} state to kick off Jenkins build? Thanks!
> Conflicting JARs org.apache.derby exist in Oozie
>
>
> Key: OOZIE-3185
> URL: https://issues.apache.org/jira/browse/OOZIE-3185
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1
>Reporter: PandaMonkey
>Assignee: PandaMonkey
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: oozie.txt
>
>
> Hi, by analyzing oozie-master\core\pom.xml, I found that there are two
> versions of org.apache.derby:derby. Their introduced paths are:
> 1.
> org.apache.oozie:oozie-core:5.0.0-beta1::null->org.apache.derby:derby:10.10.1.1::compile
> 2.
> org.apache.oozie:oozie-core:5.0.0-beta1::null->org.apache.hive:hive-exec:1.2.0:core:compile->org.apache.hive:hive-metastore:1.2.0::compile->org.apache.derby:derby:10.11.1.1::compile
> Of them, org.apache.derby:derby:10.10.1.1 is the transitively introduced by
> org.apache.hive:hive-exec and org.apache.derby:derby:10.11.1.1 is the direct
> dependency. By further analyzing the source code, *we found they have too
> many different features*.
> Also, I found that you have already gave derby a exclusion tag when
> transitively introducing derby by hive-beeline to avoid dependency conflict.
> So it would be nice to keep the consistency of library version for the
> project long-term health, maybe *update org.apache.derby:derby from 10.10.1.1
> to 10.11.1.1, or exclude derby from hive-exec* is a good choice. Hope this
> can help you. Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OOZIE-3185) Conflicting JARs org.apache.derby exist in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros reassigned OOZIE-3185: --- Assignee: PandaMonkey > Conflicting JARs org.apache.derby exist in Oozie > > > Key: OOZIE-3185 > URL: https://issues.apache.org/jira/browse/OOZIE-3185 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0b1 >Reporter: PandaMonkey >Assignee: PandaMonkey >Priority: Major > Fix For: 5.0.0 > > Attachments: oozie.txt > > > Hi, by analyzing oozie-master\core\pom.xml, I found that there are two > versions of org.apache.derby:derby. Their introduced paths are: > 1. > org.apache.oozie:oozie-core:5.0.0-beta1::null->org.apache.derby:derby:10.10.1.1::compile > 2. > org.apache.oozie:oozie-core:5.0.0-beta1::null->org.apache.hive:hive-exec:1.2.0:core:compile->org.apache.hive:hive-metastore:1.2.0::compile->org.apache.derby:derby:10.11.1.1::compile > Of them, org.apache.derby:derby:10.10.1.1 is the transitively introduced by > org.apache.hive:hive-exec and org.apache.derby:derby:10.11.1.1 is the direct > dependency. By further analyzing the source code, *we found they have too > many different features*. > Also, I found that you have already gave derby a exclusion tag when > transitively introducing derby by hive-beeline to avoid dependency conflict. > So it would be nice to keep the consistency of library version for the > project long-term health, maybe *update org.apache.derby:derby from 10.10.1.1 > to 10.11.1.1, or exclude derby from hive-exec* is a good choice. Hope this > can help you. Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3172) Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to com.fasterxml.jackson
[
https://issues.apache.org/jira/browse/OOZIE-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364081#comment-16364081
]
Andras Piros commented on OOZIE-3172:
-
Thanks [~gezapeti]! Committed to {{master}}.
> Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to
> com.fasterxml.jackson
> --
>
> Key: OOZIE-3172
> URL: https://issues.apache.org/jira/browse/OOZIE-3172
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3172.001.patch, OOZIE-3172.002.patch,
> OOZIE-3172.003.patch
>
>
> Jackson 1.9.3 is way too old, and has several security vulnerabilities as
> well. Jackson 2.9.2 covers most of these.
> Let's switch from {{org.codehaus.jackson}} to {{com.fasterxml.jackson}} in
> Oozie's direct (non-transitive) dependencies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3183) Better logging for SshActionExecutor and extended HA capability when calling to remote host
[
https://issues.apache.org/jira/browse/OOZIE-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364037#comment-16364037
]
Andras Piros commented on OOZIE-3183:
-
Failing test {{TestHiveActionExecutor#testHiveAction()}} is unrelated. FindBugs
warnings come from untouched parts. [~gezapeti] can you please review?
> Better logging for SshActionExecutor and extended HA capability when calling
> to remote host
> ---
>
> Key: OOZIE-3183
> URL: https://issues.apache.org/jira/browse/OOZIE-3183
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3183.001.patch
>
>
> In some High Availability environments, when an SSH action's
> {{SshActionExecutor#start()}} runs on another node than
> {{SshActionExecutor#check()}}, it can happen that due to network bandwidth
> reasons calling {{Process#exitValue()}} is not enough - but calling
> {{Process#waitFor()}} is better.
> Also, for better error root cause analysis, enhance logging along
> {{SshActionExecutor}} for the critical parts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
