[jira] [Assigned] (OOZIE-3339) Enhance proxy job submission documentation
[ https://issues.apache.org/jira/browse/OOZIE-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon reassigned OOZIE-3339: - Assignee: (was: Andras Salamon) > Enhance proxy job submission documentation > -- > > Key: OOZIE-3339 > URL: https://issues.apache.org/jira/browse/OOZIE-3339 > Project: Oozie > Issue Type: Improvement > Components: docs >Reporter: Andras Salamon >Priority: Minor > > The documentation should make it clear that the proxy job submission methods > can only be used for simple job submissions (e.g. no support for credentials) > and the documentation should suggest an alternative if someone wants a more > complex workflow. > Relevant parts of the documentation: > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_MapReduce_Job_Submission] > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Pig_Job_Submission] > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Hive_Job_Submission] > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Sqoop_Job_Submission] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3339) Enhance proxy job submission documentation
[ https://issues.apache.org/jira/browse/OOZIE-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-3339: -- Description: The documentation should make it clear that the proxy job submission methods can only be used for simple job submissions (e.g. no support for credentials) and the documentation should suggest an alternative if someone wants a more complex workflow. Relevant parts of the documentation: * [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_MapReduce_Job_Submission] * [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Pig_Job_Submission] * [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Hive_Job_Submission] * [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Sqoop_Job_Submission] was:The documentation should make it clear that the proxy job submission methods can only be used for simple job submissions (e.g. no support for credentials) and the documentation should suggest an alternative if someone wants a more complex workflow. > Enhance proxy job submission documentation > -- > > Key: OOZIE-3339 > URL: https://issues.apache.org/jira/browse/OOZIE-3339 > Project: Oozie > Issue Type: Improvement > Components: docs >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > > The documentation should make it clear that the proxy job submission methods > can only be used for simple job submissions (e.g. no support for credentials) > and the documentation should suggest an alternative if someone wants a more > complex workflow. > Relevant parts of the documentation: > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_MapReduce_Job_Submission] > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Pig_Job_Submission] > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Hive_Job_Submission] > * > [https://oozie.apache.org/docs/5.1.0/WebServicesAPI.html#Proxy_Sqoop_Job_Submission] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3428) [docs] Update Setting Up Oozie with an Alternate Tomcat documentation
[
https://issues.apache.org/jira/browse/OOZIE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon reassigned OOZIE-3428:
-
Assignee: (was: Andras Salamon)
> [docs] Update Setting Up Oozie with an Alternate Tomcat documentation
> -
>
> Key: OOZIE-3428
> URL: https://issues.apache.org/jira/browse/OOZIE-3428
> Project: Oozie
> Issue Type: Improvement
> Components: docs
>Affects Versions: trunk
>Reporter: Andras Salamon
>Priority: Major
> Fix For: 5.2.0
>
>
> Although Oozie uses embedded Jetty now, during compilation it still creates a
> {{war}} file which can be run in a different servlet container.
> The [current
> documentation|https://oozie.apache.org/docs/5.1.0/AG_Install.html#Setting_Up_Oozie_with_an_Alternate_Tomcat]
> does not describe the full process, we need to enhance it:
> * A full sample of the {{addtowar}} script usage.
> * Necessary {{CATALINA_OPTS}} settings.
> * A simple script to cleanup the {{webapps}}, copy the war, restart Tomcat.
> * Some info about the supported Tomcat versions would be also useful.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3455) Inconsistent CoordMaterializeTransitionXCommand logging
[
https://issues.apache.org/jira/browse/OOZIE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815468#comment-16815468
]
Andras Salamon commented on OOZIE-3455:
---
I've started to implement the log cleanup at the end of the {{XCommand}}, the
results look good, the {{Acquired lock for}} line no longer contains info about
the previous job (like user, group, and token), and at the end of the unit test
log line changes from
{noformat}
15:56:43,429 INFO Services:520 - USER[test] GROUP[testg] TOKEN[]
APP[COORD-TEST] JOB[000-190411155642913-oozie-andr-C] ACTION[]
Shutdown{noformat}
to
{noformat}
16:32:11,464 INFO Services:520 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Shutdown
{noformat}
which is more logical because the service shutdown is not connected to a job.
I've added the log cleanup to the end of the {{XCommand}} class so it affects
all the commands not only CoordMaterializeTransitionXCommand. I don't think
that this jira is the correct place to address this issue. I'd rather modify
only {{CoordMaterializeTransitionXCommand}} problems in this jira, and a
separate jira should modify {{XCommand}}. What do you think [~kmarton]?
> Inconsistent CoordMaterializeTransitionXCommand logging
>
>
> Key: OOZIE-3455
> URL: https://issues.apache.org/jira/browse/OOZIE-3455
> Project: Oozie
> Issue Type: Bug
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3455-01.patch
>
>
> Logging of {{CoordMaterializeTransitionXCommand}} lock acquiring is
> incomplete. The following four lines is printed by
> {{TestCoordMaterializeTransitionXCommand.testMaterializationLookupRelativeDays3()}}.
> The first two lines does not contain the {{USER[test] GROUP[testg] TOKEN[]
> APP[COORD-TEST] JOB[000-190327125652192-oozie-andr-C] ACTION[-]}}
> {noformat}
> 12:56:52,866 DEBUG CoordMaterializeTransitionXCommand:526 - Acquired lock for
> [000-190327125652192-oozie-andr-C] in [coord_mater]
> 12:56:52,899 DEBUG CoordMaterializeTransitionXCommand:526 - Materializing
> coord job id=000-190327125652192-oozie-andr-C, start=2019-03-27T11:56Z,
> end=2019-03-27T12:56Z, window=3600
> 12:56:52,900 DEBUG CoordMaterializeTransitionXCommand:526 - USER[test]
> GROUP[testg] TOKEN[] APP[COORD-TEST]
> JOB[000-190327125652192-oozie-andr-C] ACTION[-] Execute command
> [coord_mater] key [000-190327125652192-oozie-andr-C]
> 12:56:52,916 INFO CoordMaterializeTransitionXCommand:520 - USER[test]
> GROUP[testg] TOKEN[] APP[COORD-TEST]
> JOB[000-190327125652192-oozie-andr-C] ACTION[-] materialize actions for
> tz=Pacific Standard Time,
> {noformat}
> Sometimes the information is not missing, but contains info from a previous
> run, which is very confusing:
> {noformat}
> 6:39:57,236 DEBUG CoordMaterializeTransitionXCommand:526 - USER[test]
> GROUP[testg] TOKEN[] APP[COORD-TEST]
> JOB[000-190327163956391-oozie-andr-C] ACTION[-] Acquired lock for
> [000-190327163956963-oozie-andr-C] in [coord_mater]
> 16:39:57,252 DEBUG CoordMaterializeTransitionXCommand:526 - USER[test]
> GROUP[testg] TOKEN[] APP[COORD-TEST]
> JOB[000-190327163956391-oozie-andr-C] ACTION[-] Materializing coord job
> id=000-190327163956963-oozie-andr-C, start=2013-07-18T00:00Z,
> end=2013-07-18T01:00Z, window=3600
> 16:39:57,252 DEBUG CoordMaterializeTransitionXCommand:526 - USER[test]
> GROUP[testg] TOKEN[] APP[COORD-TEST]
> JOB[000-190327163956963-oozie-andr-C] ACTION[-] Execute command
> [coord_mater] key [000-190327163956963-oozie-andr-C]
> 16:39:57,255 INFO CoordMaterializeTransitionXCommand:520 - USER[test]
> GROUP[testg] TOKEN[] APP[COORD-TEST]
> JOB[000-190327163956963-oozie-andr-C] ACTION[-] materialize actions for
> tz=Pacific Standard Time,
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.13.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815420#comment-16815420 ] Denes Bodo commented on OOZIE-2231: --- [~kmarton] Curator 2.x supports Zookeeper 3.4.x. Curator 3.x supports only Zookeeper 3.5.x. It can be risky from Oozie side to replace the supported ZK version but using the same dependency as Hadoop means reliability. As I see in Hadoop and Hive pom files they use Curator 2.12.0 so Ozoie shall use the same in my opinion. I cannot see if any of these use Curator 3. +1 to Curator 2.12.0 or 2.13.0 > Upgrade curator to latest version 2.13.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3469) Extract common methods, fields from oozie-core to a new oozie-common module
Julia Kinga Marton created OOZIE-3469: - Summary: Extract common methods, fields from oozie-core to a new oozie-common module Key: OOZIE-3469 URL: https://issues.apache.org/jira/browse/OOZIE-3469 Project: Oozie Issue Type: Improvement Reporter: Julia Kinga Marton Assignee: Julia Kinga Marton Oozie sharelib needs oozie-core as dependency, what will bring in a lot of transitive dependencies, what may cause conflicts and we need to exclude them one by one. In a lot of cases we need only some constants from oozie-core. Let's investigate whether we can extract this common fields, (methods/classes) into a new common module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.13.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815338#comment-16815338 ] Julia Kinga Marton commented on OOZIE-2231: --- By changing the curator version to 2.13.0 (without any other change), I was able to compile the project and even all the unit tests passed. There is a high backwards compatibility between 2.5.0 and 2.13.0: [https://abi-laboratory.pro/index.php?view=timeline=java=curator-recipes] There is one removed method, and a few privacy and data type related changes, what may cause issues if we will change the version in our sharelib as well. However, Hive was able to do the upgrade with just one line change in the pom: HIVE-14069, but they changed it 3.0.0 right after it was upgraded in Hadoop. Spark also had some issues with upgrading it: SPARK-20884 > Upgrade curator to latest version 2.13.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2422) Recovery service loads jobs which doesn't need recovery
[
https://issues.apache.org/jira/browse/OOZIE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815233#comment-16815233
]
Hadoop QA commented on OOZIE-2422:
--
Testing JIRA OOZIE-2422
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Recovery service loads jobs which doesn't need recovery
> ---
>
> Key: OOZIE-2422
> URL: https://issues.apache.org/jira/browse/OOZIE-2422
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Satish Subhashrao Saley
>Priority: Major
> Attachments: OOZIE-2422-1.patch
>
>
> {code}
> @NamedQuery(name = "GET_COORD_ACTIONS_FOR_RECOVERY_OLDER_THAN", query
> = "select a.id, a.jobId, a.statusStr, a.externalId, a.pending from
> CoordinatorActionBean a where a.pending > 0 AND (a.statusStr = 'SUSPENDED' OR
> a.statusStr = 'KILLED' OR a.statusStr = 'RUNNING') AND
> a.lastModifiedTimestamp <= :lastModifiedTime"),
> {code}
> Recovery service use above sql to recover killed/suspended/running action and
> in code it checks for external id. Checking of externalId can be done in sql
> itself.
> {code}
> else if (caction.getStatus() == CoordinatorActionBean.Status.SUSPENDED) {
> if (caction.getExternalId() != null &&
> caction.getPending() > 1) {
> queueCallable(new
> SuspendXCommand(caction.getExternalId()));
> log.debug("Recover a SUSPENDED coord action
> and resubmit SuspendXCommand :"
> + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.KILLED) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> KillXCommand(caction.getExternalId()));
> log.debug("Recover a KILLED coord action and
> resubmit KillXCommand :" + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.RUNNING) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> ResumeXCommand(caction.getExternalId()));
> log.debug("Recover a RUNNING coord action and
> resubmit ResumeXCommand :" + caction.getId());
> }
> }
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-2422 PreCommit Build #1085
Jira: https://issues.apache.org/jira/browse/OOZIE-2422 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1085/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 803.16 KB...] [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/main/resources [INFO] Copying 3 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ oozie-zookeeper-security-tests --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ oozie-zookeeper-security-tests --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/resources [INFO] Copying 3 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ oozie-zookeeper-security-tests --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 2 source files to /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/target/test-classes [INFO] /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java: Some input files use or override a deprecated API. [INFO] /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java: Recompile with -Xlint:deprecation for details. [INFO] [INFO] >>> spotbugs-maven-plugin:3.1.11:check (default-cli) > :spotbugs @ oozie-zookeeper-security-tests >>> [INFO] [INFO] --- spotbugs-maven-plugin:3.1.11:spotbugs (spotbugs) @ oozie-zookeeper-security-tests --- [INFO] [INFO] <<< spotbugs-maven-plugin:3.1.11:check (default-cli) < :spotbugs @ oozie-zookeeper-security-tests <<< [INFO] [INFO] [INFO] --- spotbugs-maven-plugin:3.1.11:check (default-cli) @ oozie-zookeeper-security-tests --- [INFO] [INFO] Reactor Summary for Apache Oozie Main 5.2.0-SNAPSHOT: [INFO] [INFO] Apache Oozie Main .. SUCCESS [ 3.404 s] [INFO] Apache Oozie Fluent Job SUCCESS [ 0.077 s] [INFO] Apache Oozie Fluent Job API SUCCESS [ 31.835 s] [INFO] Apache Oozie Client SUCCESS [ 17.819 s] [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 14.296 s] [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ 7.535 s] [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ 5.642 s] [INFO] Apache Oozie Core .. SUCCESS [01:39 min] [INFO] Apache Oozie Share Lib Streaming ... SUCCESS [ 8.365 s] [INFO] Apache Oozie Share Lib Pig . SUCCESS [ 9.044 s] [INFO] Apache Oozie Share Lib Git . SUCCESS [ 9.007 s] [INFO] Apache Oozie Share Lib Hive SUCCESS [ 8.911 s] [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 9.797 s] [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [ 6.414 s] [INFO] Apache Oozie Examples .. SUCCESS [ 11.228 s] [INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 11.633 s] [INFO] Apache Oozie Share Lib . SUCCESS [ 0.020 s] [INFO] Apache Oozie Docs .. SUCCESS [ 3.598 s] [INFO] Apache Oozie WebApp SUCCESS [ 8.792 s] [INFO] Apache Oozie Tools . SUCCESS [ 15.123 s] [INFO] Apache Oozie MiniOozie . SUCCESS [ 1.808 s] [INFO] Apache Oozie Fluent Job Client . SUCCESS [ 0.442 s] [INFO] Apache Oozie Server SUCCESS [ 10.750 s] [INFO] Apache Oozie Distro SUCCESS [ 2.335 s] [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ 3.028 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 05:02 min [INFO] Finished at: 2019-04-11T09:09:51Z [INFO] [TRACE] SpotBugs output in HEAD verified and saved Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Applying patch Checking patch
[jira] [Commented] (OOZIE-2422) Recovery service loads jobs which doesn't need recovery
[
https://issues.apache.org/jira/browse/OOZIE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815226#comment-16815226
]
Hadoop QA commented on OOZIE-2422:
--
PreCommit-OOZIE-Build started
> Recovery service loads jobs which doesn't need recovery
> ---
>
> Key: OOZIE-2422
> URL: https://issues.apache.org/jira/browse/OOZIE-2422
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Satish Subhashrao Saley
>Priority: Major
> Attachments: OOZIE-2422-1.patch
>
>
> {code}
> @NamedQuery(name = "GET_COORD_ACTIONS_FOR_RECOVERY_OLDER_THAN", query
> = "select a.id, a.jobId, a.statusStr, a.externalId, a.pending from
> CoordinatorActionBean a where a.pending > 0 AND (a.statusStr = 'SUSPENDED' OR
> a.statusStr = 'KILLED' OR a.statusStr = 'RUNNING') AND
> a.lastModifiedTimestamp <= :lastModifiedTime"),
> {code}
> Recovery service use above sql to recover killed/suspended/running action and
> in code it checks for external id. Checking of externalId can be done in sql
> itself.
> {code}
> else if (caction.getStatus() == CoordinatorActionBean.Status.SUSPENDED) {
> if (caction.getExternalId() != null &&
> caction.getPending() > 1) {
> queueCallable(new
> SuspendXCommand(caction.getExternalId()));
> log.debug("Recover a SUSPENDED coord action
> and resubmit SuspendXCommand :"
> + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.KILLED) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> KillXCommand(caction.getExternalId()));
> log.debug("Recover a KILLED coord action and
> resubmit KillXCommand :" + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.RUNNING) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> ResumeXCommand(caction.getExternalId()));
> log.debug("Recover a RUNNING coord action and
> resubmit ResumeXCommand :" + caction.getId());
> }
> }
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.13.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-2231: -- Summary: Upgrade curator to latest version 2.13.0 (was: Upgrade curator to latest version 2.12.0) > Upgrade curator to latest version 2.13.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2422) Recovery service loads jobs which doesn't need recovery
[
https://issues.apache.org/jira/browse/OOZIE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815219#comment-16815219
]
Andras Salamon commented on OOZIE-2422:
---
I think we should keep the {{a.pending > 0}} condition for the KILLED and
RUNNING state. If we put the checking into the SQL, we could simplify the Java
code.
> Recovery service loads jobs which doesn't need recovery
> ---
>
> Key: OOZIE-2422
> URL: https://issues.apache.org/jira/browse/OOZIE-2422
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Satish Subhashrao Saley
>Priority: Major
> Attachments: OOZIE-2422-1.patch
>
>
> {code}
> @NamedQuery(name = "GET_COORD_ACTIONS_FOR_RECOVERY_OLDER_THAN", query
> = "select a.id, a.jobId, a.statusStr, a.externalId, a.pending from
> CoordinatorActionBean a where a.pending > 0 AND (a.statusStr = 'SUSPENDED' OR
> a.statusStr = 'KILLED' OR a.statusStr = 'RUNNING') AND
> a.lastModifiedTimestamp <= :lastModifiedTime"),
> {code}
> Recovery service use above sql to recover killed/suspended/running action and
> in code it checks for external id. Checking of externalId can be done in sql
> itself.
> {code}
> else if (caction.getStatus() == CoordinatorActionBean.Status.SUSPENDED) {
> if (caction.getExternalId() != null &&
> caction.getPending() > 1) {
> queueCallable(new
> SuspendXCommand(caction.getExternalId()));
> log.debug("Recover a SUSPENDED coord action
> and resubmit SuspendXCommand :"
> + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.KILLED) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> KillXCommand(caction.getExternalId()));
> log.debug("Recover a KILLED coord action and
> resubmit KillXCommand :" + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.RUNNING) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> ResumeXCommand(caction.getExternalId()));
> log.debug("Recover a RUNNING coord action and
> resubmit ResumeXCommand :" + caction.getId());
> }
> }
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2927) Append new line character for Hive2 query using query tag
[ https://issues.apache.org/jira/browse/OOZIE-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815195#comment-16815195 ] Andras Salamon commented on OOZIE-2927: --- [~abhishekbafna] Do you mind if I take this over? > Append new line character for Hive2 query using query tag > - > > Key: OOZIE-2927 > URL: https://issues.apache.org/jira/browse/OOZIE-2927 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna >Priority: Major > Attachments: OOZIE-2927-00.patch, OOZIE-2927-01.patch > > > Without the new line character in the query, beeline does not do anything. > Though hive2 action completes successfully but query is not executed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-2231) Upgrade curator to latest version 2.12.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815142#comment-16815142 ] Julia Kinga Marton edited comment on OOZIE-2231 at 4/11/19 7:54 AM: In the meantime apache curator 2.13 has been released, so I will try to upgrade to this version. was (Author: kmarton): In the meantime apache curato 2.13 has been released, so I will try to upgrade to this version. > Upgrade curator to latest version 2.12.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.12.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815142#comment-16815142 ] Julia Kinga Marton commented on OOZIE-2231: --- IN the meantime apache curato 2.13 has been released, so I will try to upgrade to this version. > Upgrade curator to latest version 2.12.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-2231) Upgrade curator to latest version 2.12.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815142#comment-16815142 ] Julia Kinga Marton edited comment on OOZIE-2231 at 4/11/19 7:20 AM: In the meantime apache curato 2.13 has been released, so I will try to upgrade to this version. was (Author: kmarton): IN the meantime apache curato 2.13 has been released, so I will try to upgrade to this version. > Upgrade curator to latest version 2.12.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Julia Kinga Marton >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (91 issues) Subscriber: ooziedaily Key Summary OOZIE-3468 Use modernizer plugin https://issues.apache.org/jira/browse/OOZIE-3468 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3455 Inconsistent CoordMaterializeTransitionXCommand logging https://issues.apache.org/jira/browse/OOZIE-3455 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator
