[jira] [Commented] (OOZIE-1603) cannot submit job to oozie on mysql
[
https://issues.apache.org/jira/browse/OOZIE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13814748#comment-13814748
]
Hadoop QA commented on OOZIE-1603:
--
Testing JIRA OOZIE-1603
Cleaning local svn workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:red}-1{color} the patch modifies 2 JPA file(s), persistence.xml or
*-orm.xml
{color:green}+1 TESTS{color}
.Tests run: 1350
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/876/
cannot submit job to oozie on mysql
---
Key: OOZIE-1603
URL: https://issues.apache.org/jira/browse/OOZIE-1603
Project: Oozie
Issue Type: Bug
Affects Versions: trunk
Reporter: Bowen Zhang
Assignee: Virag Kothari
Priority: Blocker
Attachments: OOZIE-1603.patch
job submission throws the following exception:
org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error
in operation, openjpa-2.2.2-r422266:1468616 fatal store error
org.apache.openjpa.persistence.RollbackException: The transaction has been
rolled back. See the nested exceptions for details on the errors that
occurred.
at
org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:252)
at
org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:72)
at org.apache.oozie.command.XCommand.call(XCommand.java:280)
at org.apache.oozie.DagEngine.submitJob(DagEngine.java:109)
at
org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:188)
at
org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:88)
at
org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at
org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:286)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:126)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:84)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
[jira] [Commented] (OOZIE-1603) cannot submit job to oozie on mysql
[ https://issues.apache.org/jira/browse/OOZIE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13815035#comment-13815035 ] Rohini Palaniswamy commented on OOZIE-1603: --- property name=openjpa.jdbc.DBDictionary value=ClobTypeName =MEDIUMTEXT/ - let's leave this as is even though we changed all clob columns to blob. If for any reason we had to add one this might be missed. Another thing, can you make oozie.output.compression.codec value as none as default in oozie-default.xml. cannot submit job to oozie on mysql --- Key: OOZIE-1603 URL: https://issues.apache.org/jira/browse/OOZIE-1603 Project: Oozie Issue Type: Bug Affects Versions: trunk Reporter: Bowen Zhang Assignee: Virag Kothari Priority: Blocker Attachments: OOZIE-1603.patch job submission throws the following exception: org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:252) at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:72) at org.apache.oozie.command.XCommand.call(XCommand.java:280) at org.apache.oozie.DagEngine.submitJob(DagEngine.java:109) at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:188) at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:88) at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99) at javax.servlet.http.HttpServlet.service(HttpServlet.java:643) at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:286) at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:126) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372) at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:84) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:680) Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:231) ... 29 more Caused by: org.apache.oozie.executor.jpa.JPAExecutorException: E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.service.JPAService.executeBatchInsertUpdateDelete(JPAService.java:392) at
[jira] [Created] (OOZIE-1605) Add common custom filter applied to Wf/Coord/Bundle jobs on oozie UI
Ryota Egashira created OOZIE-1605: - Summary: Add common custom filter applied to Wf/Coord/Bundle jobs on oozie UI Key: OOZIE-1605 URL: https://issues.apache.org/jira/browse/OOZIE-1605 Project: Oozie Issue Type: Bug Reporter: Ryota Egashira Assignee: Ryota Egashira 1. add common custom filter which applied to all types of jobs (wf/coord/bundle), especially user name, since customer only want to see his/her jobs but not others. store it into cookie not to lose it after browser refresh. 2. make current custom filter to be stored in cookie, as well. -- This message was sent by Atlassian JIRA (v6.1#6144)
Review Request 15276: [OOZIE-1605] Add common custom filter applied to Wf/Coord/Bundle jobs on oozie UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15276/ --- Review request for oozie. Bugs: OOZIE-1605 https://issues.apache.org/jira/browse/OOZIE-1605 Repository: oozie Description --- https://issues.apache.org/jira/browse/OOZIE-1605 Diffs - trunk/webapp/src/main/webapp/oozie-console.js 1538031 Diff: https://reviews.apache.org/r/15276/diff/ Testing --- locally tested Thanks, Ryota Egashira
[jira] [Updated] (OOZIE-1603) cannot submit job to oozie on mysql
[ https://issues.apache.org/jira/browse/OOZIE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated OOZIE-1603: - Attachment: OOZIE-1603.patch Updated patch turns off compression by default and adds the clob property back cannot submit job to oozie on mysql --- Key: OOZIE-1603 URL: https://issues.apache.org/jira/browse/OOZIE-1603 Project: Oozie Issue Type: Bug Affects Versions: trunk Reporter: Bowen Zhang Assignee: Virag Kothari Priority: Blocker Attachments: OOZIE-1603.patch, OOZIE-1603.patch job submission throws the following exception: org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:252) at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:72) at org.apache.oozie.command.XCommand.call(XCommand.java:280) at org.apache.oozie.DagEngine.submitJob(DagEngine.java:109) at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:188) at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:88) at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99) at javax.servlet.http.HttpServlet.service(HttpServlet.java:643) at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:286) at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:126) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372) at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:84) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:680) Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:231) ... 29 more Caused by: org.apache.oozie.executor.jpa.JPAExecutorException: E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.service.JPAService.executeBatchInsertUpdateDelete(JPAService.java:392) at org.apache.oozie.executor.jpa.BatchQueryExecutor.executeBatchInsertUpdateDelete(BatchQueryExecutor.java:140) at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:228) ... 29 more Caused by: openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The
[jira] [Commented] (OOZIE-1603) cannot submit job to oozie on mysql
[ https://issues.apache.org/jira/browse/OOZIE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13815141#comment-13815141 ] Rohini Palaniswamy commented on OOZIE-1603: --- +1 cannot submit job to oozie on mysql --- Key: OOZIE-1603 URL: https://issues.apache.org/jira/browse/OOZIE-1603 Project: Oozie Issue Type: Bug Affects Versions: trunk Reporter: Bowen Zhang Assignee: Virag Kothari Priority: Blocker Attachments: OOZIE-1603.patch, OOZIE-1603.patch job submission throws the following exception: org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:252) at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:72) at org.apache.oozie.command.XCommand.call(XCommand.java:280) at org.apache.oozie.DagEngine.submitJob(DagEngine.java:109) at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:188) at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:88) at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99) at javax.servlet.http.HttpServlet.service(HttpServlet.java:643) at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:286) at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:126) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372) at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:84) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:680) Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:231) ... 29 more Caused by: org.apache.oozie.executor.jpa.JPAExecutorException: E0603: SQL error in operation, openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.oozie.service.JPAService.executeBatchInsertUpdateDelete(JPAService.java:392) at org.apache.oozie.executor.jpa.BatchQueryExecutor.executeBatchInsertUpdateDelete(BatchQueryExecutor.java:140) at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:228) ... 29 more Caused by: openjpa-2.2.2-r422266:1468616 fatal store error org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested
[jira] [Commented] (OOZIE-1581) Workflow performance optimizations
[ https://issues.apache.org/jira/browse/OOZIE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13815219#comment-13815219 ] Virag Kothari commented on OOZIE-1581: -- Couple of things discussed with Mona: OOZIE-243 adds the control nodes to db and all the control nodes follow the command life cycle. Instead of optimizing the execution of commands for this nodes (async to sync), can we devise a way such that the control nodes can be configured to be not added as wf_actions in database? In this case, they wont go through the command life cycle like action nodes. While synchronously calling command, it would be good to check if we can pass the bean object to another command directly, so the other command doesn't need to perform db operations like loadState() and verifyPreCondition(). E.g CoordActionReady to CoordActionStart in the patch. Also, please check whether this type of optimization can be done at other places. Workflow performance optimizations -- Key: OOZIE-1581 URL: https://issues.apache.org/jira/browse/OOZIE-1581 Project: Oozie Issue Type: Improvement Components: workflow Affects Versions: trunk Reporter: Mona Chitnis Assignee: Mona Chitnis Fix For: trunk Attachments: OOZIE-1581-2.patch, OOZIE-1581.patch Creating a combo JIRA for small performance optimizations. 1. changing from asynchronous action start to a synchronous one, to overcome the undue delay in transitioning from ::start:: control node to the actual first node, owing to a loaded queue. This delay has been observed to be close to 30 min at times in stress conditions. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (OOZIE-1603) cannot submit job to oozie on mysql
[
https://issues.apache.org/jira/browse/OOZIE-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13815276#comment-13815276
]
Hadoop QA commented on OOZIE-1603:
--
Testing JIRA OOZIE-1603
Cleaning local svn workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:red}-1{color} the patch modifies 2 JPA file(s), persistence.xml or
*-orm.xml
{color:green}+1 TESTS{color}
.Tests run: 1350
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/877/
cannot submit job to oozie on mysql
---
Key: OOZIE-1603
URL: https://issues.apache.org/jira/browse/OOZIE-1603
Project: Oozie
Issue Type: Bug
Affects Versions: trunk
Reporter: Bowen Zhang
Assignee: Virag Kothari
Priority: Blocker
Attachments: OOZIE-1603.patch, OOZIE-1603.patch
job submission throws the following exception:
org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error
in operation, openjpa-2.2.2-r422266:1468616 fatal store error
org.apache.openjpa.persistence.RollbackException: The transaction has been
rolled back. See the nested exceptions for details on the errors that
occurred.
at
org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:252)
at
org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:72)
at org.apache.oozie.command.XCommand.call(XCommand.java:280)
at org.apache.oozie.DagEngine.submitJob(DagEngine.java:109)
at
org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:188)
at
org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:88)
at
org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at
org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:286)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:126)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:84)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
Re: Review Request 14741: Setup sharelib using script and pickup latest(honor ship.launcher) and remove DFS dependency at startup.
On Nov. 5, 2013, 10:59 p.m., Rohini Palaniswamy wrote:
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java,
line 97
https://reviews.apache.org/r/14741/diff/6/?file=376492#file376492line97
It does not need launcherlibPath. Needs the system lib path only.
Also log a error message if exception in purge. don't fail
exception is handled and logged at parent method.
On Nov. 5, 2013, 10:59 p.m., Rohini Palaniswamy wrote:
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java,
line 287
https://reviews.apache.org/r/14741/diff/6/?file=376492#file376492line287
how does this work without prefix?
This will be taken care by..
PathFilter directoryFilter = new PathFilter() {
@Override
public boolean accept(Path path) {
return path.getName().startsWith(prefix);
}
};
On Nov. 5, 2013, 10:59 p.m., Rohini Palaniswamy wrote:
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java,
line 252
https://reviews.apache.org/r/14741/diff/6/?file=376492#file376492line252
Do purge irrespective of the setting.
If property is not set, then why to delete something and throw exception.
On Nov. 5, 2013, 10:59 p.m., Rohini Palaniswamy wrote:
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java,
line 257
https://reviews.apache.org/r/14741/diff/6/?file=376492#file376492line257
Do purge irrespective of the setting.
If property is not set, then why to delete something and throw exception.
On Nov. 5, 2013, 10:59 p.m., Rohini Palaniswamy wrote:
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java,
line 128
https://reviews.apache.org/r/14741/diff/6/?file=376492#file376492line128
This code should not be removed
Discussed with Viraj, we need this method.
details @ https://issues.apache.org/jira/browse/OOZIE-1528
- Purshotam
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14741/#review28053
---
On Nov. 7, 2013, 5:59 a.m., Purshotam Shah wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14741/
---
(Updated Nov. 7, 2013, 5:59 a.m.)
Review request for oozie.
Bugs: OOZIE-1584
https://issues.apache.org/jira/browse/OOZIE-1584
Repository: oozie
Description
---
Supported features.
-
1. If Oozie.action.ship.launcher.jar = true then, Oozie auto-deploy jar at
server start.
2. Admin copies sharelib to new timestamp directory and issues invalidate
command to Oozie server. Oozie picks latest shared lib.
3. Oozie server doesn't auto deploy sharelib at startup. It picks the latest
sharelib jar pushed by admin(based on timestamp).
4. Sharelib mapping files.
Sharelib mapping file can be also configured.Configured file is a key value
mapping, where key will be the action and value is DFS directory of sharelib
files for action.
This can be configured in oozie-site.xml as :
property
nameoozie.service.ShareLibService.mapping.file/name
value/value
description
Sharelib mapping files contains list of key=value,
where key is the action key and value is the DFS location of
sharelib files.
/description
/property
DFS after generating launcher jar by ooozie server and sharelib by cli
makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls
/user/purushah/share/lib/
/user/purushah/share/lib/launcher_20131017092254
/user/purushah/share/lib/launcher_20131017093814
/user/purushah/share/lib/launcher_20131017094652
/user/purushah/share/lib/launcher_20131017094836
/user/purushah/share/lib/launcher_20131017095549
/user/purushah/share/lib/lib_20131017092806
makebag-lm:example purushah$
Purging.
-
There are two set( launcher_ for launcher jars and Lib_ for shared lib) of
directory and purging happens for both at startup.
Anything older than ( defined in property) is purged.
We always keep 2 set of directory, purging happens if number of sharelib
directory 2.
Multiple version.
---
To support multiple version, sharelib will be in below format.
Sharelib will cache all versions and return list of dfs files for a particular
version.
makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls
/user/purushah/share/lib/
Found 15 items
/user/purushah/share/lib/launcher_20131017092254
Re: Review Request 14741: Setup sharelib using script and pickup latest(honor ship.launcher) and remove DFS dependency at startup.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14741/ --- (Updated Nov. 7, 2013, 5:59 a.m.) Review request for oozie. Changes --- Review changes. Bugs: OOZIE-1584 https://issues.apache.org/jira/browse/OOZIE-1584 Repository: oozie Description --- Supported features. - 1. If Oozie.action.ship.launcher.jar = true then, Oozie auto-deploy jar at server start. 2. Admin copies sharelib to new timestamp directory and issues invalidate command to Oozie server. Oozie picks latest shared lib. 3. Oozie server doesn't auto deploy sharelib at startup. It picks the latest sharelib jar pushed by admin(based on timestamp). 4. Sharelib mapping files. Sharelib mapping file can be also configured.Configured file is a key value mapping, where key will be the action and value is DFS directory of sharelib files for action. This can be configured in oozie-site.xml as : property nameoozie.service.ShareLibService.mapping.file/name value/value description Sharelib mapping files contains list of key=value, where key is the action key and value is the DFS location of sharelib files. /description /property DFS after generating launcher jar by ooozie server and sharelib by cli makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls /user/purushah/share/lib/ /user/purushah/share/lib/launcher_20131017092254 /user/purushah/share/lib/launcher_20131017093814 /user/purushah/share/lib/launcher_20131017094652 /user/purushah/share/lib/launcher_20131017094836 /user/purushah/share/lib/launcher_20131017095549 /user/purushah/share/lib/lib_20131017092806 makebag-lm:example purushah$ Purging. - There are two set( launcher_ for launcher jars and Lib_ for shared lib) of directory and purging happens for both at startup. Anything older than ( defined in property) is purged. We always keep 2 set of directory, purging happens if number of sharelib directory 2. Multiple version. --- To support multiple version, sharelib will be in below format. Sharelib will cache all versions and return list of dfs files for a particular version. makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls /user/purushah/share/lib/ Found 15 items /user/purushah/share/lib/launcher_20131017092254 /user/purushah/share/lib/launcher_20131017093814 /user/purushah/share/lib/launcher_20131017094652 /user/purushah/share/lib/launcher_20131017094836 /user/purushah/share/lib/launcher_20131017095549 /user/purushah/share/lib/lib_20131017092806/pig/... /user/purushah/share/lib/lib_20131017092806/pig_9/... /user/purushah/share/lib/lib_20131017092806/pig_10/... makebag-lm:example purushah$ Diffs (updated) - http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1532127 http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java 1532127 http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml 1532127 http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/service/TestShareLibService.java 1532127 http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/AG_Install.twiki 1532127 http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/DG_QuickStart.twiki 1532127 http://svn.apache.org/repos/asf/oozie/trunk/tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 1532127 Diff: https://reviews.apache.org/r/14741/diff/ Testing --- Thanks, Purshotam Shah
