[
https://issues.apache.org/jira/browse/VFS-343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ralph Goers closed VFS-343.
---------------------------
Resolution: Fixed
Fix Version/s: 2.0
The fix has been validated. In addition, the other locking changes have greatly
improved performance although bottlenecks are still being seen.
> Deadlock occurs in HTTP or WebDAV when all the connections are in use.
> ----------------------------------------------------------------------
>
> Key: VFS-343
> URL: https://issues.apache.org/jira/browse/VFS-343
> Project: Commons VFS
> Issue Type: Bug
> Reporter: Ralph Goers
> Fix For: 2.0
>
>
> The stacktrace below shows several threads all waiting on a FileSystem lock.
> The thread holding that lock is waiting for an HTTP connection from the
> connection pool. However, each of the threads waiting on the FileSystem lock
> has already acquired a connection from the pool. This issue can be resolved
> by reducing the amount of locking that occurs on the FileSystem objects. In
> this case the threads are attempting to get the lock to increment a counter.
> Changing the counters to AtomicIntegers will resolve the problem.
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (11.0-b16 mixed mode):
> "Attach Listener" daemon prio=10 tid=0x0000000053834400 nid=0x23b1 waiting on
> condition [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "MultiThreadedHttpConnectionManager cleanup" daemon prio=10
> tid=0x00000000538ff400 nid=0x21f0 in Object.wait()
> [0x000000004077e000..0x000000004077ec10]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aab82694428> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x00002aab82694428> (a java.lang.ref.ReferenceQueue$Lock)
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1082)
> "Thread 2" prio=10 tid=0x00002aabc00a0800 nid=0x21ef waiting for monitor
> entry [0x0000000040d17000..0x0000000040d17c90]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
> - waiting to lock <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
> at
> org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at java.lang.Thread.run(Thread.java:619)
> "Thread 3" prio=10 tid=0x00002aabc009f800 nid=0x21ec in Object.wait()
> [0x0000000042593000..0x0000000042593d10]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aab826c67a8> (a
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool)
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:509)
> - locked <0x00002aab826c67a8> (a
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool)
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:394)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:152)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.execute(WebdavFileObject.java:349)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperties(WebdavFileObject.java:442)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperties(WebdavFileObject.java:417)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperty(WebdavFileObject.java:410)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperty(WebdavFileObject.java:402)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.isDirectory(WebdavFileObject.java:382)
> at
> org.apache.commons.vfs.provider.webdav.WebdavFileObject.doGetType(WebdavFileObject.java:112)
> at
> org.apache.commons.vfs.provider.AbstractFileObject.getType(AbstractFileObject.java:441)
> - locked <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.AbstractFileObject.exists(AbstractFileObject.java:424)
> at
> org.apache.commons.configuration.VFSFileSystem.locateFromURL(VFSFileSystem.java:293)
> at
> org.apache.commons.configuration.ConfigurationUtils.locate(ConfigurationUtils.java:466)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:245)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at java.lang.Thread.run(Thread.java:619)
> "Thread 4" prio=10 tid=0x00002aabc00ce000 nid=0x21eb waiting for monitor
> entry [0x0000000042492000..0x0000000042492d90]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
> - waiting to lock <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
> at
> org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at java.lang.Thread.run(Thread.java:619)
> "Thread 5" prio=10 tid=0x00002aabc00d2000 nid=0x21ea waiting for monitor
> entry [0x0000000042391000..0x0000000042391a10]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
> - waiting to lock <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
> at
> org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at java.lang.Thread.run(Thread.java:619)
> "Thread 6" prio=10 tid=0x00002aabc00af400 nid=0x21e9 waiting for monitor
> entry [0x0000000042290000..0x0000000042290a90]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.commons.vfs.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:294)
> - waiting to lock <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:86)
> at
> org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:66)
> at
> org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:692)
> at
> org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:648)
> at
> org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:604)
> at
> org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:99)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at java.lang.Thread.run(Thread.java:619)
> "Thread 7" prio=10 tid=0x00002aabc001cc00 nid=0x21e8 waiting for monitor
> entry [0x000000004218f000..0x000000004218fb10]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
> - waiting to lock <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
> at
> org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at java.lang.Thread.run(Thread.java:619)
> "org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread" daemon
> prio=10 tid=0x00002aabc001ec00 nid=0x21e7 in Object.wait()
> [0x000000004208e000..0x000000004208eb90]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaabe1f0198> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x00002aaabe1f0198> (a java.lang.ref.ReferenceQueue$Lock)
> at
> org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread.run(SoftRefFilesCache.java:79)
> "Low Memory Detector" daemon prio=10 tid=0x000000005360bc00 nid=0x21e4
> runnable [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "CompilerThread1" daemon prio=10 tid=0x0000000053609000 nid=0x21e3 waiting on
> condition [0x0000000000000000..0x0000000041d8a4c0]
> java.lang.Thread.State: RUNNABLE
> "CompilerThread0" daemon prio=10 tid=0x0000000053605400 nid=0x21e2 waiting on
> condition [0x0000000000000000..0x000000004165c5d0]
> java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x0000000053603400 nid=0x21e1 runnable
> [0x0000000000000000..0x0000000040ed66a0]
> java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x00000000535e0c00 nid=0x21e0 in Object.wait()
> [0x000000004155c000..0x000000004155ca90]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaabe1f0278> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x00002aaabe1f0278> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
> "Reference Handler" daemon prio=10 tid=0x00000000535df000 nid=0x21df in
> Object.wait() [0x000000004145b000..0x000000004145bb10]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00002aaabe1f2370> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0x00002aaabe1f2370> (a java.lang.ref.Reference$Lock)
> "Thread 1" prio=10 tid=0x0000000053427400 nid=0x21d5 waiting for monitor
> entry [0x00000000412af000..0x00000000412afed0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
> - waiting to lock <0x00002aab827f92c8> (a
> org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
> at
> org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
> at
> org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
> at
> org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
> at
> org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
> at
> com.diginsite.services.configsync.ConfigurationSynchronizer.main(ConfigurationSynchronizer.java:1790)
> "VM Thread" prio=10 tid=0x00000000535d9800 nid=0x21de runnable
> "GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000053432000 nid=0x21d6
> runnable
> "GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000053433800 nid=0x21d7
> runnable
> "GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000053435400 nid=0x21d8
> runnable
> "GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000053436c00 nid=0x21d9
> runnable
> "GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000053438400 nid=0x21da
> runnable
> "GC task thread#5 (ParallelGC)" prio=10 tid=0x000000005343a000 nid=0x21db
> runnable
> "GC task thread#6 (ParallelGC)" prio=10 tid=0x000000005343b800 nid=0x21dc
> runnable
> "GC task thread#7 (ParallelGC)" prio=10 tid=0x000000005343d000 nid=0x21dd
> runnable
> "VM Periodic Task Thread" prio=10 tid=0x000000005360e000 nid=0x21e5 waiting
> on condition
> JNI global references: 902
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
