If it happens again, try wrapping your task logic in a try/finally and put this
in the finally block as a safety net:
ERXEC.unlockAllContextsForCurrentThread();
On Feb 13, 2012, at 1:03 AM, Alexis Tual wrote:
> Looks like ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIds is involved in
> some way. I've replaced it with ec.faultForGlobalID(gid, ec) and so far no
> deadlock (finger crossed).
>
> Alex
>
> 2012/2/13 Alexis Tual <[email protected]>
> Thanks for your quick answer.
> None of the two threads are launched by ERXExecutorService. One is managed by
> Quartz, the other one is in fact a TimerTask so launched in a periodical
> manner...
> There's hope thought, as thanks to the tracing
> (er.extensions.ERXApplication.traceOpenEditingContextLocks...) I've found a
> suspect :
>
> Outstanding at @Thread[QuartzScheduler_Worker-1,5,main]
> java.lang.Exception: Locked
> at er.extensions.eof.ERXEC.traceLock(ERXEC.java:541)
> at er.extensions.eof.ERXEC.lock(ERXEC.java:511)
> at
> org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.switchEcForValidationAndPurge(JobValidationReport.java:143)
> at
> org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:81)
> at
> org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
> java.lang.Exception: Locked
> at er.extensions.eof.ERXEC.traceLock(ERXEC.java:541)
> at er.extensions.eof.ERXEC.lock(ERXEC.java:511)
> at
> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:243)
> at
> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:229)
> at
> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectWithGlobalID(ERXEOGlobalIDUtilities.java:209)
> at
> org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:68)
> at
> org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
> ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIds is still locking the ec...
> The first stack is normal as I recycle the ec's (create a new one, unlock the
> previous, lock the new one)
>
> Alex
>
>
> 2012/2/13 Kieran Kelleher <[email protected]>
> How are you dispatching the background threads? Are you using
> ERXExecutorService.executorService() to execute/submit Runnable/Callable?
>
> On Feb 12, 2012, at 9:17 PM, Alexis Tual wrote:
>
>> Hi Kieran and others,
>> related to this topic, I have a deadlock situation which I can't explain.
>> I have 2 background threads doing EOF stuff, the first one is doing a lot of
>> fetches and can take several minutes. The second one is doing periodical
>> short work (a fetch every 30sec).
>> For these two threads I use manual locking : locking at the begining of the
>> processing and unlocking at the end. I don't share ec's or eo's between
>> threads. However they use the same OSC.
>>
>> Here's the stack trace, any help is welcomed !
>>
>> Found one Java-level deadlock:
>> =============================
>> "Timer-1":
>> waiting for ownable synchronizer 7de742608, (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync),
>> which is held by "QuartzScheduler_Worker-1"
>> "QuartzScheduler_Worker-1":
>> waiting for ownable synchronizer 7de40c8e8, (a
>> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync),
>> which is held by "Timer-1"
>>
>> Java stack information for the threads listed above:
>> ===================================================
>> "Timer-1":
>> at sun.misc.Unsafe.park(Native Method)
>> - parking to wait for <7de742608> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
>> at
>> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
>> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
>> at
>> com.webobjects.eocontrol.EOObjectStoreCoordinator.lock(EOObjectStoreCoordinator.java:420)
>> at
>> com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4666)
>> at er.extensions.eof.ERXEC.lockObjectStore(ERXEC.java:724)
>> at
>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4067)
>> at er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1215)
>> at
>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444)
>> at
>> er.extensions.eof.ERXFetchSpecification.fetchObjects(ERXFetchSpecification.java:125)
>> at
>> org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$DefaultDelegate.fetchEvents(FwkCktlEvenementPrincipal.java:321)
>> at
>> org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$SchedulingEvenementTask.rescheduleAllEvents(FwkCktlEvenementPrincipal.java:374)
>> at
>> org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$SchedulingEvenementTask.run(FwkCktlEvenementPrincipal.java:364)
>> at java.util.TimerThread.mainLoop(Timer.java:512)
>> at java.util.TimerThread.run(Timer.java:462)
>> "QuartzScheduler_Worker-1":
>> at sun.misc.Unsafe.park(Native Method)
>> - parking to wait for <7de40c8e8> (a
>> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
>> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
>> at
>> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
>> at
>> com.webobjects.eocontrol.EOSharedEditingContext.lock(EOSharedEditingContext.java:736)
>> at
>> com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4664)
>> at er.extensions.eof.ERXEC.lockObjectStore(ERXEC.java:724)
>> at
>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4067)
>> at er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1215)
>> at
>> com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444)
>> at
>> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:290)
>> at
>> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:229)
>> at
>> er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectWithGlobalID(ERXEOGlobalIDUtilities.java:209)
>> at
>> org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:67)
>> at
>> org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116)
>> at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
>> at
>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
>>
>> Found 1 deadlock.
>>
>>
>>
>>
>>
>> 2012/2/13 Kieran Kelleher <[email protected]>
>> Just to follow up..... you can use the task OSC pool as follows:
>>
>> osc = ERXTaskObjectStoreCoordinatorPool.objectStoreCoordinator()
>>
>> You can configure how many OSC's in the pool using this property which
>> defaults to 1 (which is still better than using the default OSC for
>> background EOF tasks):
>>
>>
>> er.extensions.concurrency.ERXTaskObjectStoreCoordinatorPool.maxCoordinators
>> = 1
>>
>> As a convenience, there is an abstract "task" class,ERXAbstractTask, you can
>> subclass which has a newEditingContext() method that incorporates the OSC
>> pool for you and adjusts EC timestamp to ensure fresh EOs in background
>> tasks. IIRC, this is all explained in the WOWODc presentation too.
>>
>>
>> On Feb 11, 2012, at 6:00 PM, Kieran Kelleher wrote:
>>
>> > Don't worry about the OSC unless you manipulating the OSC directly.
>> >
>> > BTW intensive background EOF activity can impact regular user EOF
>> > performance. One approach is to use a dedicated OSC pool for background
>> > tasks. Such a pool exists in Wonder. IIRC it is used in the WOWODC example
>> > app.
>> >
>> > Regards, Kieran.
>> > (Sent from my iPhone)
>> >
>> >
>> > On Feb 11, 2012, at 7:56 AM, Giles Palmer <[email protected]> wrote:
>> >
>> >> Hi
>> >>
>> >> Just to extend this... when (if at all) should we be locking the
>> >> EOObjectStoreCoordinator as well, in the context of background threads?
>> >>
>> >> I lock and unlock my ec in a try, catch, finally, is that enough or do I
>> >> also need to worry about the OSC?
>> >>
>> >> Thanks
>> >>
>> >>
>> >> Giles
>> >>
>> >>
>> >>> Also if you use ERXExecutorService to execute any Plain Old Java
>> >>> Callable (or Runnable), your editing contexts will be auto unlocked by
>> >>> safety-net unlocker at the end of execution if you haven't done so
>> >>> ....... however it is highly recommend that you follow the ec
>> >>> lock/try/finally/unlock pattern in in your Callable.call() or
>> >>> Runnable.run() methods anyway which will be better for long running
>> >>> tasks, recycling ec's if needed, etc. it doesn't hurt.
>> >>>
>> >>> EOEditingContext ec = ERXEC.newEditingContext();
>> >>> ec.lock();
>> >>> try {
>> >>> ........
>> >>> } finally {
>> >>> ec.unlock();
>> >>> }
>> >>>
>> >>> And yeah, do what Ramsey said .... save yourself time figuring out
>> >>> concurrency management of EC's, etc. by watching the WOWODC presentation
>> >>> from 2011. There is a bunch of stuff related to this in Wonder to make
>> >>> life in the background easy and totally painless for you and that WOWODC
>> >>> session explains it's usage.
>> >>>
>> >>>
>> >>> On Feb 10, 2012, at 11:34 AM, Ramsey Gurley wrote:
>> >>>
>> >>>> If you use ERXRunnable, then you still get autolocking. Just don't try
>> >>>> to pass an existing EC or EOs to a background thread. Pass EOs by
>> >>>> global id and create the EC on the thread.
>> >>>>
>> >>>> See also Kieran's most recent WOWODC presentation on ERXExecutor stuff
>> >>>> and background thread processing.
>> >>>>
>> >>>> Ramsey
>> >>>>
>> >>>> On Feb 10, 2012, at 9:30 AM, Michael Gargano wrote:
>> >>>>
>> >>>>> Hi everyone,
>> >>>>>
>> >>>>> I just want to get clarification on something before I get myself
>> >>>>> into trouble later. If I have a headless WO app (or potential just a
>> >>>>> spawned worker thread), and I'm using ERXEC, I need to manually lock
>> >>>>> and unlock the context, correct? I'm assuming that ERXEC does the
>> >>>>> autolock/autounlock in the RR loop, which I won't have in this
>> >>>>> situation.
>> >>>>>
>> >>>>> Thanks.
>> >>>>> -Mike
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Do not post admin requests to the list. They will be ignored.
>> >>>>> Webobjects-dev mailing list ([email protected])
>> >>>>> Help/Unsubscribe/Update your Subscription:
>> >>>>> https://lists.apple.com/mailman/options/webobjects-dev/rgurley%40smarthealth.com
>> >>>>>
>> >>>>> This email sent to [email protected]
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Do not post admin requests to the list. They will be ignored.
>> >>>> Webobjects-dev mailing list ([email protected])
>> >>>> Help/Unsubscribe/Update your Subscription:
>> >>>> https://lists.apple.com/mailman/options/webobjects-dev/kelleherk%40gmail.com
>> >>>>
>> >>>> This email sent to [email protected]
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Do not post admin requests to the list. They will be ignored.
>> >>> Webobjects-dev mailing list ([email protected])
>> >>> Help/Unsubscribe/Update your Subscription:
>> >>> https://lists.apple.com/mailman/options/webobjects-dev/lists%40cedarstone.co.uk
>> >>>
>> >>> This email sent to [email protected]
>> >>
>>
>>
>> _______________________________________________
>> Do not post admin requests to the list. They will be ignored.
>> Webobjects-dev mailing list ([email protected])
>> Help/Unsubscribe/Update your Subscription:
>> https://lists.apple.com/mailman/options/webobjects-dev/alexis.tual%40gmail.com
>>
>> This email sent to [email protected]
>>
>
>
>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list ([email protected])
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to [email protected]
