[
https://issues.apache.org/jira/browse/JCR-929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499116
]
Ian Boston commented on JCR-929:
--------------------------------
There is a stack trace of a node locked in waiting, the HTTP are locked and
every request to this node, that hits the http thread will block in the same
wait pattern
Starting Thread Monitor ==================
Thread Transient File Reaper waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.jackrabbit.util.TransientFileFactory$ReaperThread.run(TransientFileFactory.java:148)
Thread TP-Processor3 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread TP-Processor2 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread TP-Processor1 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor25 waiting by [EMAIL PROTECTED] ::WAITING at
EDU.oswego.cs.dl.util.concurrent.ReentrantLock.acquire(null:-1)
at java.lang.Object.wait(Object.java:-2)
at java.lang.Object.wait(Object.java:474)
at EDU.oswego.cs.dl.util.concurrent.ReentrantLock.acquire(null:-1)
at
org.apache.jackrabbit.core.lock.LockManagerImpl.acquire(LockManagerImpl.java:599)
at
org.apache.jackrabbit.core.lock.LockManagerImpl.externalLock(LockManagerImpl.java:973)
at
org.apache.jackrabbit.core.cluster.ClusterNode.process(ClusterNode.java:723)
at
org.apache.jackrabbit.core.cluster.ClusterNode.consume(ClusterNode.java:910)
at
org.apache.jackrabbit.core.journal.AbstractJournal.doSync(AbstractJournal.java:191)
at
org.apache.jackrabbit.core.journal.AbstractJournal.lockAndSync(AbstractJournal.java:241)
at
org.apache.jackrabbit.core.journal.DefaultRecordProducer.append(DefaultRecordProducer.java:51)
at
org.apache.jackrabbit.core.cluster.ClusterNode$WorkspaceUpdateChannel.updateCreated(ClusterNode.java:466)
at
org.apache.jackrabbit.core.state.SharedItemStateManager$Update.begin(SharedItemStateManager.java:530)
at
org.apache.jackrabbit.core.state.SharedItemStateManager.beginUpdate(SharedItemStateManager.java:825)
at
org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:855)
at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
at
org.apache.jackrabbit.core.state.XAItemStateManager.update(XAItemStateManager.java:313)
at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:302)
at
org.apache.jackrabbit.core.state.SessionItemStateManager.update(SessionItemStateManager.java:306)
at org.apache.jackrabbit.core.ItemImpl.save(ItemImpl.java:1214)
at org.apache.jackrabbit.core.NodeImpl.lock(NodeImpl.java:4070)
at
org.apache.jackrabbit.webdav.simple.DavResourceImpl.lock(DavResourceImpl.java:685)
at
org.apache.jackrabbit.server.AbstractWebdavServlet.doLock(AbstractWebdavServlet.java:689)
at
org.apache.jackrabbit.server.AbstractWebdavServlet.execute(AbstractWebdavServlet.java:259)
at
org.apache.jackrabbit.server.AbstractWebdavServlet.service(AbstractWebdavServlet.java:193)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:613)
Thread http-8580-Processor24 waiting by [EMAIL PROTECTED] ::WAITING at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(null:-1)
at java.lang.Object.wait(Object.java:-2)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(null:-1)
at
org.apache.jackrabbit.core.journal.AbstractJournal.lockAndSync(AbstractJournal.java:228)
at
org.apache.jackrabbit.core.journal.DefaultRecordProducer.append(DefaultRecordProducer.java:51)
at
org.apache.jackrabbit.core.cluster.ClusterNode$WorkspaceLockChannel.unlocked(ClusterNode.java:637)
at
org.apache.jackrabbit.core.lock.LockManagerImpl.internalUnlock(LockManagerImpl.java:338)
at
org.apache.jackrabbit.core.lock.LockManagerImpl.unlock(LockManagerImpl.java:428)
at
org.apache.jackrabbit.core.lock.XALockManager.unlock(XALockManager.java:103)
at org.apache.jackrabbit.core.NodeImpl.unlock(NodeImpl.java:4133)
at
org.apache.jackrabbit.webdav.simple.DavResourceImpl.unlock(DavResourceImpl.java:739)
at
org.apache.jackrabbit.server.AbstractWebdavServlet.doUnlock(AbstractWebdavServlet.java:710)
at
org.apache.jackrabbit.server.AbstractWebdavServlet.execute(AbstractWebdavServlet.java:262)
at
org.apache.jackrabbit.server.AbstractWebdavServlet.service(AbstractWebdavServlet.java:193)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:613)
Thread http-8580-Processor22 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor21 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor20 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor19 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor18 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor17 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor16 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor15 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor14 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor13 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor12 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor11 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor10 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor9 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor8 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor7 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor6 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor5 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor4 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor3 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor2 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread http-8580-Processor1 waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
Thread ClusterNode-node2 waiting by [EMAIL PROTECTED] ::WAITING at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock.acquire(null:-1)
at java.lang.Object.wait(Object.java:-2)
at java.lang.Object.wait(Object.java:474)
at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock.acquire(null:-1)
at
org.apache.jackrabbit.core.journal.AbstractJournal.sync(AbstractJournal.java:160)
at
org.apache.jackrabbit.core.cluster.ClusterNode.sync(ClusterNode.java:283)
at org.apache.jackrabbit.core.cluster.ClusterNode.run(ClusterNode.java:254)
at java.lang.Thread.run(Thread.java:613)
Thread IndexMerger waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.commons.collections.buffer.BlockingBuffer.remove(BlockingBuffer.java:107)
Thread IndexMerger waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.commons.collections.buffer.BlockingBuffer.remove(BlockingBuffer.java:107)
Thread ObservationManager waiting by [EMAIL PROTECTED] ::WAITING at
org.apache.commons.collections.buffer.BlockingBuffer.remove(BlockingBuffer.java:107)
Thread Finalizer waiting by [EMAIL PROTECTED] ::WAITING
Thread Reference Handler waiting by [EMAIL PROTECTED] ::WAITING
Done Thread Monitor ==================
> Under Heavy load in a Cluster HTTP Threads Block and stall requests
> -------------------------------------------------------------------
>
> Key: JCR-929
> URL: https://issues.apache.org/jira/browse/JCR-929
> Project: Jackrabbit
> Issue Type: Bug
> Components: core
> Affects Versions: 1.3
> Environment: 2 Node Cluster, OSX, JDK 1.5 with DatabaseJournal,
> DatabasePersistanceManager, all content in DB, using WebDAV to load
> Reporter: Ian Boston
> Assigned To: Dominique Pfister
>
> Under Heavy load created by mounting both nodes in the cluster in OSX Finder
> and then uploading large numebers of files to each node at the same time ( a
> few 1000), eventually one of the nodes stops responding and the Finder mount
> timesout and disconnects.
> Once that happens that node becomes unusable.
> More mount attempts will prompt for a password indicating HTTP is still
> running, but will timeout once the connection is authenticated.
> Access by the Web Browser will prompt for a password, conenct and provide a
> once only listing of any collection in the workspace. If you try to refresh
> that collection, the HTTP request hangs forever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
