Exceptions coming due to DB transaction are resolved in latest releases. This exceptions might not make your job failed.
However, the exception that Mona (thanks! good find) mentioned will surely make your job failed. Are you setting the variable "inputData" in coordinator.xml? If yes, (most possibly) you dataset's initial instance is the same as the job's start-time? Please look at this : http://incubator.apache.org/oozie/docs/3.1.3/docs/CoordinatorFunctionalSpec.html#a6.6.9._Dataset_Instance_Resolution_for_Instances_Before_the_Initial_Instance if things are not clear, please send us the coordinator xml definition. Regards, Mohammad ________________________________ From: Mona Chitnis <[email protected]> To: "[email protected]" <[email protected]>; "[email protected]" <[email protected]> Cc: "[email protected]" <[email protected]>; "[email protected]" <[email protected]> Sent: Friday, June 8, 2012 3:47 PM Subject: Re: Issue related optimistic lock violation in Oozie while executing coordinator workflow Hi, I also happened to this see this Warning in the bottom part of your logs ACTION[0000004-120608174022095-oozie-oozi-W@java-node] ELException in ActionStartCommand javax.servlet.jsp.el.ELException: variable [inputData] cannot be resolved Can you retry by fixing this first? --Mona On 6/8/12 6:18 AM, "Sukirti Panigrahi" <[email protected]> wrote: > >Hi > >We are trying to create a coordinator workflow on oozie-2.3.2+27.19.But >we >are facing a certain problem,if the number of >coordinator exceeding more than two.We are getting the following error > > 2012-06-08 18:39:19,315 INFO WfEndCommand:525 - USER[sukirti] >GROUP[users] TOKEN[] APP[java-main-logs] >JOB[0000004-120608174022095-oozie-oozi-W] >ACTION[0000004-120608174022095-oozie-oozi-W@java-node] XException - >connection logs from store conn 597321988, false 2012-06-08 18:39:19,317 >ERROR ActionStartCommand:522 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] >ACTION[-] XException, org.apache.oozie.command.CommandException: E0607: >Other error in operation [getWorkflow], An optimistic lock violation was >detected when flushing object instance "A lock could not be obtained >within >the time requested [java.lang.String]" to the data store. This indicates >that the object was concurrently modified in another transaction. >org.apache.oozie.command.CommandException: E0607: Other error in >operation >[getWorkflow], An optimistic lock violation was detected when flushing >object instance "A lock could not be obtained within the time requested >[java.lang.String]" to the data store. This indicates that the object >was >concurrently modified in another transaction. at >org.apache.oozie.command.Command.call(Command.java:259) at >org.apache.oozie.command.wf.ActionStartCommand.handleError(ActionStartComm >an >d.java:241) at >org.apache.oozie.command.wf.ActionStartCommand.call(ActionStartCommand.jav >a: >108) at >org.apache.oozie.command.wf.ActionStartCommand.execute(ActionStartCommand. >ja >va:251) at >org.apache.oozie.command.wf.ActionStartCommand.execute(ActionStartCommand. >ja >va:47) at org.apache.oozie.command.Command.call(Command.java:202) >at >org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Callable >Qu >eueService.java:132) at >java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor. >ja >va:886) at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:9 >08) at java.lang.Thread.run(Thread.java:662) Caused by: >org.apache.oozie.store.StoreException: E0607: Other error in operation >[getWorkflow], An optimistic lock violation was detected when flushing >object instance "A lock could not be obtained within the time requested >[java.lang.String]" to the data store. This indicates that the object >was >concurrently modified in another transaction. at >org.apache.oozie.store.WorkflowStore.doOperation(WorkflowStore.java:823) > at >org.apache.oozie.store.WorkflowStore.getWorkflow(WorkflowStore.java:125) > at org.apache.oozie.command.wf.WfEndCommand.call(WfEndCommand.java:56) > at >org.apache.oozie.command.wf.WfEndCommand.call(WfEndCommand.java:42) at >org.apache.oozie.command.Command.execute(Command.java:586) at >org.apache.oozie.command.Command.call(Command.java:202) ... 9 more >Caused >by: <openjpa-1.2.1-r752877:753278 nonfatal store error> >org.apache.openjpa.persistence.OptimisticLockException: An optimistic >lock >violation was detected when flushing object instance "A lock could not be >obtained within the time requested [java.lang.String]" to the data store. > >This indicates that the object was concurrently modified in another >transaction. at >org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4228) > at >org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.ja >va >:4197) at >org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) > > at >org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:88) > at >org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:64) > at >org.apache.openjpa.jdbc.kernel.SelectResultObjectProvider.handleCheckedExc >ep >tion(SelectResultObjectProvider.java:155) at >org.apache.openjpa.lib.rop.EagerResultList.<init>(EagerResultList.java:40) > > at org.apache.openjpa.kernel.QueryImpl.toResult(QueryImpl.java:1228) >at >org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:990) at >org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:848) at >org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:779) at >org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:525 >) > at org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:254) > at >org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:294) > > at >org.apache.oozie.store.WorkflowStore.getWorkflowOnly(WorkflowStore.java:83 >8) > at >org.apache.oozie.store.WorkflowStore.access$100(WorkflowStore.java:54) > at org.apache.oozie.store.WorkflowStore$2.call(WorkflowStore.java:128) > at >org.apache.oozie.store.WorkflowStore$2.call(WorkflowStore.java:125) at >org.apache.oozie.store.WorkflowStore.doOperation(WorkflowStore.java:808) > ... 14 more Caused by: java.sql.SQLTransactionRollbackException: A lock >could not be obtained within the time requested at >org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown >Source) at >org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown >Source) at >org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unkn >ow >n Source) at >org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown > >Source) at >org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown >Source) > at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown >Source) at >org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown >Source) at >org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown >Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown >Source) > at >org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java: >20 >7) at >org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java: >20 >7) at >org.apache.openjpa.lib.jdbc.DelegatingResultSet.next(DelegatingResultSet.j >av >a:106) at >org.apache.openjpa.jdbc.sql.ResultSetResult.nextInternal(ResultSetResult.j >av >a:222) at >org.apache.openjpa.jdbc.sql.SelectImpl$SelectResult.nextInternal(SelectImp >l. >java:2457) at >org.apache.openjpa.jdbc.sql.AbstractResult.next(AbstractResult.java:173) > at >org.apache.openjpa.jdbc.kernel.SelectResultObjectProvider.next(SelectResul >tO >bjectProvider.java:99) at >org.apache.openjpa.lib.rop.EagerResultList.<init>(EagerResultList.java:35) > > ... 26 more Caused by: java.sql.SQLException: A lock could not be >obtained >within the time requested at >org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown >Source) at >org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcros >sD >RDA(Unknown Source) ... 43 more Caused by: ERROR 40XL1: A lock could not >be obtained within the time requested at >org.apache.derby.iapi.error.StandardException.newException(Unknown >Source) > at >org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown >Source) at >org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObj >ec >t(Unknown Source) at >org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(U >nk >nown Source) at >org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject >(U >nknown Source) at >org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(U >nk >nown Source) at >org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown >Source) at >org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown >Source) at >org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPag >e( >Unknown Source) at >org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow >(U >nknown Source) at >org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow >(U >nknown Source) at >org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(Unknow >n >Source) at >org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(Unknown >Source) at >org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unkno >wn > Source) at >org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Un >kn >own Source) at >org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCor >e( >Unknown Source) at >org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore( >Un >known Source) at >org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore( >Un >known Source) at >org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unkno >wn > Source) ... 36 more 2012-06-08 18:39:19,318 INFO >ActionStartCommand:525 >- USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] XException - >connection >logs from store conn 1304116788, false 2012-06-08 18:39:19,330 WARN >CallableQueueService$CallableWrapper:528 - USER[-] GROUP[-] TOKEN[-] >APP[-] >JOB[-] ACTION[-] exception callable [action.start], E0607: Other error in >operation [getWorkflow], An optimistic lock violation was detected when >flushing object instance "A lock could not be obtained within the time >requested [java.lang.String]" to the data store. This indicates that the >object was concurrently modified in another transaction. >org.apache.oozie.command.CommandException: E0607: Other error in >operation >[getWorkflow], An optimistic lock violation was detected when flushing >object instance "A lock could not be obtained within the time requested >[java.lang.String]" to the data store. This indicates that the object >was >concurrently modified in another transaction. at >org.apache.oozie.command.Command.call(Command.java:259) at >org.apache.oozie.command.wf.ActionStartCommand.handleError(ActionStartComm >an >d.java:241) at >org.apache.oozie.command.wf.ActionStartCommand.call(ActionStartCommand.jav >a: >108) at >org.apache.oozie.command.wf.ActionStartCommand.execute(ActionStartCommand. >ja >va:251) at >org.apache.oozie.command.wf.ActionStartCommand.execute(ActionStartCommand. >ja >va:47) at org.apache.oozie.command.Command.call(Command.java:202) >at >org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Callable >Qu >eueService.java:132) at >java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor. >ja >va:886) at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:9 >08) at java.lang.Thread.run(Thread.java:662) Caused by: >org.apache.oozie.store.StoreException: E0607: Other error in operation >[getWorkflow], An optimistic lock violation was detected when flushing >object instance "A lock could not be obtained within the time requested >[java.lang.String]" to the data store. This indicates that the object >was >concurrently modified in another transaction. at >org.apache.oozie.store.WorkflowStore.doOperation(WorkflowStore.java:823) > at >org.apache.oozie.store.WorkflowStore.getWorkflow(WorkflowStore.java:125) > at org.apache.oozie.command.wf.WfEndCommand.call(WfEndCommand.java:56) > at >org.apache.oozie.command.wf.WfEndCommand.call(WfEndCommand.java:42) at >org.apache.oozie.command.Command.execute(Command.java:586) at >org.apache.oozie.command.Command.call(Command.java:202) ... 9 more >Caused >by: <openjpa-1.2.1-r752877:753278 nonfatal store error> >org.apache.openjpa.persistence.OptimisticLockException: An optimistic >lock >violation was detected when flushing object instance "A lock could not be >obtained within the time requested [java.lang.String]" to the data store. > >This indicates that the object was concurrently modified in another >transaction. at >org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4228) > at >org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.ja >va >:4197) at >org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) > > at >org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:88) > at >org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:64) > at >org.apache.openjpa.jdbc.kernel.SelectResultObjectProvider.handleCheckedExc >ep >tion(SelectResultObjectProvider.java:155) at >org.apache.openjpa.lib.rop.EagerResultList.<init>(EagerResultList.java:40) > > at org.apache.openjpa.kernel.QueryImpl.toResult(QueryImpl.java:1228) >at >org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:990) at >org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:848) at >org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:779) at >org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:525 >) > at org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:254) > at >org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:294) > > at >org.apache.oozie.store.WorkflowStore.getWorkflowOnly(WorkflowStore.java:83 >8) > at >org.apache.oozie.store.WorkflowStore.access$100(WorkflowStore.java:54) > at org.apache.oozie.store.WorkflowStore$2.call(WorkflowStore.java:128) > at >org.apache.oozie.store.WorkflowStore$2.call(WorkflowStore.java:125) at >org.apache.oozie.store.WorkflowStore.doOperation(WorkflowStore.java:808) > ... 14 more Caused by: java.sql.SQLTransactionRollbackException: A lock >could not be obtained within the time requested at >org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown >Source) at >org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown >Source) at >org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unkn >ow >n Source) at >org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown > >Source) at >org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown >Source) > at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown >Source) at >org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown >Source) at >org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown >Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown >Source) > at >org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java: >20 >7) at >org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java: >20 >7) at >org.apache.openjpa.lib.jdbc.DelegatingResultSet.next(DelegatingResultSet.j >av >a:106) at >org.apache.openjpa.jdbc.sql.ResultSetResult.nextInternal(ResultSetResult.j >av >a:222) at >org.apache.openjpa.jdbc.sql.SelectImpl$SelectResult.nextInternal(SelectImp >l. >java:2457) at >org.apache.openjpa.jdbc.sql.AbstractResult.next(AbstractResult.java:173) > at >org.apache.openjpa.jdbc.kernel.SelectResultObjectProvider.next(SelectResul >tO >bjectProvider.java:99) at >org.apache.openjpa.lib.rop.EagerResultList.<init>(EagerResultList.java:35) > > ... 26 more Caused by: java.sql.SQLException: A lock could not be >obtained >within the time requested at >org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown >Source) at >org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcros >sD >RDA(Unknown Source) ... 43 more Caused by: ERROR 40XL1: A lock could not >be obtained within the time requested at >org.apache.derby.iapi.error.StandardException.newException(Unknown >Source) > at >org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown >Source) at >org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObj >ec >t(Unknown Source) at >org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(U >nk >nown Source) at >org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject >(U >nknown Source) at >org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(U >nk >nown Source) at >org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown >Source) at >org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown >Source) at >org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPag >e( >Unknown Source) at >org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow >(U >nknown Source) at >org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow >(U >nknown Source) at >org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(Unknow >n >Source) at >org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(Unknown >Source) at >org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unkno >wn > Source) at >org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Un >kn >own Source) at >org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCor >e( >Unknown Source) at >org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore( >Un >known Source) at >org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore( >Un >known Source) at >org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unkno >wn > Source) ... 36 more 2012-06-08 18:39:19,334 WARN >ActionStartCommand:528 >- USER[sukirti] GROUP[users] TOKEN[] APP[java-main-logs] >JOB[0000004-120608174022095-oozie-oozi-W] ACTION[-] >[***0000004-120608174022095-oozie-oozi-W@java-node***]In >call()....status=PREP 2012-06-08 18:39:19,343 WARN >ActionStartCommand:528 >- USER[sukirti] GROUP[users] TOKEN[] APP[java-main-logs] >JOB[0000004-120608174022095-oozie-oozi-W] >ACTION[0000004-120608174022095-oozie-oozi-W@java-node] ELException in >ActionStartCommand javax.servlet.jsp.el.ELException: variable >[inputData] >cannot be resolved at >org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java >:1 >02) at org.apache.commons.el.NamedValue.evaluate(NamedValue.java:124) > at >org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114) > > at >org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluator >Im >pl.java:274) at >org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluator >Im >pl.java:190) at >org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:199) at >org.apache.oozie.command.wf.ActionStartCommand.call(ActionStartCommand.jav >a: >94) at >org.apache.oozie.command.wf.ActionStartCommand.execute(ActionStartCommand. >ja >va:251) at >org.apache.oozie.command.wf.ActionStartCommand.execute(ActionStartCommand. >ja >va:47) at org.apache.oozie.command.Command.call(Command.java:202) >at >org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Callable >Qu >eueService.java:132) at >java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor. >ja >va:886) at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:9 >08) at java.lang.Thread.run(Thread.java:662) 2012-06-08 18:39:19,343 >WARN >ActionStartCommand:528 - USER[sukirti] GROUP[users] TOKEN[] >APP[java-main-logs] JOB[0000004-120608174022095-oozie-oozi-W] >ACTION[0000004-120608174022095-oozie-oozi-W@java-node] Failing Job due to >failed action [java-node] 2012-06-08 18:39:19,345 WARN >LiteWorkflowInstance:528 - USER[sukirti] GROUP[users] TOKEN[] >APP[java-main-logs] JOB[0000004-120608174022095-oozie-oozi-W] >ACTION[0000004-120608174022095-oozie-oozi-W@java-node] Workflow Failed. >Failing node [java-node] > >Can you please help in finding the solution to this particular problem. >Thanks and RegardsSukirti Panigrahi
