I have a highly multhreaded app...We use about 300 threads which are constantly using 100% of the CPU (its designed to do this to make efficient use of the hardware)...
I've noticed two bugs with dbcp 1.2.1...
The first is that if you use a shallow pool with a high number of threads it will lock at startup. I've also noticed this with older versions of dbcp but theres a workaround. I just set initialSize to the maxActive size of the pool.. that way all the connections are allocated in a single thread.
Now I've noticed a bad deadlock after running for about 7 hours.
I did a killall -3 to get the running stacktrace.
They all seem to be locked :
org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
Which is:
152: protected synchronized PooledConnectionAndInfo
153: getPooledConnectionAndInfo(String username, String password)
154: throws SQLException {155: if (pool == null) {156: try {157: registerPool(username, password);
158: } catch (NamingException e) {159: throw new SQLNestedException("RegisterPool failed", e);160: }
161: }
162: 163: PooledConnectionAndInfo info = null;
164: try {165: info = (PooledConnectionAndInfo) pool
166: .borrowObject(getUserPassKey(username, password));
167: }
168: catch (Exception e) {169: throw new SQLNestedException(
170: "Could not retrieve connection info from pool", e);
180: }
181: return info;
190: }
I used a tool I wrote to remove duplicate stack traces to see if there was any obvious contention and I didn't see any:
http://www.peerfear.org/rss/permalink/2003/10/26/MiningForExceptionsInLogFiles/
Total unique exception results: 24
Total matches: 370
-------------
References: 225
-------------
at org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
- waiting to lock <0x49ed74d8> (a org.apache.commons.dbcp.datasources.SharedPoolDataSource)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:631)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:615)
at org.apache.torque.Torque.getConnection(Torque.java:936)
at org.apache.torque.Torque.getConnection(Torque.java:870)
at ksa.om.BaseResourcePeer.doSelectPSLazy(BaseResourcePeer.java:1370)
at ksa.om.BaseResourcePeer.doSelect(BaseResourcePeer.java:1103)
at ksa.om.BaseResourcePeer.doSelectFirstRecordAsString(BaseResourcePeer.java:1285)
at ksa.om.ResourcePeer.getResourceAsString(ResourcePeer.java:92)
at ksa.om.Feed.getResourceAsString(Feed.java:257)
at ksa.robot.FeedTask.setFeed(FeedTask.java:138)
at ksa.robot.UpdateFeedQueue.visit(UpdateFeedQueue.java:103)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:256)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 47
-------------
at ksa.robot.TaskQueue.getNextXToProcess(TaskQueue.java:59)
- waiting to lock <0x4f9cdb00> (a ksa.robot.ArticleQueue)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:252)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 39
-------------
at ksa.robot.TaskQueue.getNextXToProcess(TaskQueue.java:61)
- waiting to lock <0x5060dfb0> (a ksa.robot.ImageQueue)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:252)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 11
-------------
at org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
- waiting to lock <0x49ed74d8> (a org.apache.commons.dbcp.datasources.SharedPoolDataSource)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:631)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:615)
at org.apache.torque.Torque.getConnection(Torque.java:936)
at ksa.om.BaseArticlePeer.retrieveByPK(BaseArticlePeer.java:813)
at ksa.om.BaseArticlePeer.retrieveByPK(BaseArticlePeer.java:795)
at ksa.robot.ArticleTask.run(ArticleTask.java:70)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:257)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 9
-------------
at ksa.robot.TaskQueue.getNextXToProcess(TaskQueue.java:59)
- waiting to lock <0x50615bb8> (a ksa.robot.InitFeedQueue)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:252)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 7
-------------
at org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
- waiting to lock <0x49ed74d8> (a org.apache.commons.dbcp.datasources.SharedPoolDataSource)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:631)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:615)
at org.apache.torque.Torque.getConnection(Torque.java:936)
at org.apache.torque.Torque.getConnection(Torque.java:870)
at ksa.om.BaseResourcePeer.doSelectPSLazy(BaseResourcePeer.java:1370)
at ksa.om.BaseResourcePeer.doSelect(BaseResourcePeer.java:1103)
at ksa.om.BaseResourcePeer.doSelectFirstRecord(BaseResourcePeer.java:1209)
at ksa.om.ResourcePeer.getResourceByValue(ResourcePeer.java:47)
at ksa.om.DuplicateResourceManager.getObject(ResourcePeer.java:102)
at ksa.om.DuplicateRecordManager.fetch(DuplicateRecordManager.java:51)
at ksa.om.ResourcePeer.getResourceByValueOrNewRecord(ResourcePeer.java:79)
at ksa.om.Feed.setLink(Feed.java:71)
at ksa.robot.FeedTaskParserListener.onChannel(FeedTaskParserListener.java:180)
at org.apache.commons.feedparser.RSSFeedParser.doParseChannel(RSSFeedParser.java:108)
at org.apache.commons.feedparser.RSSFeedParser.parse(RSSFeedParser.java:63)
at org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:102)
at org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:61)
at org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:76)
at ksa.robot.FeedTask.doFeedParse(FeedTask.java:449)
at ksa.robot.FeedTask.doUpdate(FeedTask.java:395)
at ksa.robot.FeedTask.run(FeedTask.java:170)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:257)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 6
-------------
at org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
- waiting to lock <0x49ed74d8> (a org.apache.commons.dbcp.datasources.SharedPoolDataSource)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:631)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:615)
at org.apache.torque.Torque.getConnection(Torque.java:936)
at org.apache.torque.Torque.getConnection(Torque.java:870)
at ksa.om.BaseResourcePeer.doSelectPSLazy(BaseResourcePeer.java:1370)
at ksa.om.BaseResourcePeer.doSelect(BaseResourcePeer.java:1103)
at ksa.om.BaseResourcePeer.doSelectFirstRecord(BaseResourcePeer.java:1209)
at ksa.om.BaseResourcePeer.retrieveByPK(BaseResourcePeer.java:833)
at ksa.om.BaseResourcePeer.retrieveByPK(BaseResourcePeer.java:803)
at ksa.om.BaseFeed.getResourceRelatedByResourceId(BaseFeed.java:3078)
at ksa.om.Feed.getResource(Feed.java:55)
at ksa.robot.FeedTask.run(FeedTask.java:168)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:257)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 2
-------------
at org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
- waiting to lock <0x49ed74d8> (a org.apache.commons.dbcp.datasources.SharedPoolDataSource)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:631)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:615)
at org.apache.torque.Torque.getConnection(Torque.java:936)
at ksa.om.BaseResourcePeer.retrieveByPK(BaseResourcePeer.java:802)
at ksa.om.BaseWeblog.getResource(BaseWeblog.java:1284)
at ksa.robot.ThumbnailTask.setWeblog(ThumbnailTask.java:75)
at ksa.robot.ThumbnailQueue.visit(ThumbnailQueue.java:50)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:256)
at ksa.robot.TaskThread.run(TaskThread.java:88)
-------------
References: 1
-------------
at org.apache.commons.dbcp.datasources.SharedPoolDataSource.getPooledConnectionAndInfo(SharedPoolDataSource.java:155)
- waiting to lock <0x49ed74d8> (a org.apache.commons.dbcp.datasources.SharedPoolDataSource)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:631)
at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:615)
at org.apache.torque.Torque.getConnection(Torque.java:936)
at org.apache.torque.Torque.getConnection(Torque.java:870)
at ksa.om.BaseArticlePeer.doSelectPSLazy(BaseArticlePeer.java:2075)
at ksa.om.BaseArticlePeer.doSelect(BaseArticlePeer.java:1738)
at ksa.om.ArticlePeer.getArticlesForArticleTask(ArticlePeer.java:269)
at ksa.om.ArticlePeer.getArticlesForArticleTask(ArticlePeer.java:222)
at ksa.robot.ArticleQueue.refresh(ArticleQueue.java:31)
at ksa.robot.TaskQueue.getNextXToProcess(TaskQueue.java:67)
- locked <0x4f9cdb00> (a ksa.robot.ArticleQueue)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:252)
at ksa.robot.TaskThread.run(TaskThread.java:88)
... any idea?
--
Please reply using PGP.
http://peerfear.org/pubkey.asc NewsMonster - http://www.newsmonster.org/
Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
AIM/YIM - sfburtonator, Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
