[jira] [Commented] (CONNECTORS-100) DB lock timeout, and/or indefinite or excessive database activity
[
https://issues.apache.org/jira/browse/CONNECTORS-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133845#comment-13133845
]
Karl Wright commented on CONNECTORS-100:
I have an RSS load test and a filesystem-based load test for Postgresql now,
and I hoped to use those to show success, but unfortunately I ran into new
problems.
First, the cleanup for both of these on trunk unaccountably hangs once in a
while on my laptop. The hang is a cold hang, with nothing spinning and nothing
in the log. I've created a ticket CONNECTORS-279 to figure out what's going on
here and fixing it.
Second, the RSS test did not even finish one time in the allotted interval,
even though it *should* be a good deal faster than the file system test. A lot
of queries longer than 60 seconds were reported, none of which make much sense,
since as far as queries are concerned the RSS test should be even doing fewer
of them. I haven't yet opened a ticket for this behavior.
DB lock timeout, and/or indefinite or excessive database activity
-
Key: CONNECTORS-100
URL: https://issues.apache.org/jira/browse/CONNECTORS-100
Project: ManifoldCF
Issue Type: Bug
Components: Framework core
Affects Versions: ManifoldCF 0.1, ManifoldCF 0.2, ManifoldCF 0.3,
ManifoldCF 0.4
Environment: Running unmodified dist/example from trunk/ using the
default configuration.
Reporter: Andrzej Bialecki
Assignee: Karl Wright
Fix For: ManifoldCF 0.4
When a job is started and running (via crawler-ui) occasionally it's not
possible to display a list of running jobs. The problem persists even after
restarting ACF. The following exception is thrown in the console:
{code}
org.apache.acf.core.interfaces.ACFException: Database exception: Exception
doing query: A lock could not be obtained within the time requested
at
org.apache.acf.core.database.Database.executeViaThread(Database.java:421)
at
org.apache.acf.core.database.Database.executeUncachedQuery(Database.java:465)
at
org.apache.acf.core.database.Database$QueryCacheExecutor.create(Database.java:1072)
at
org.apache.acf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
at
org.apache.acf.core.database.Database.executeQuery(Database.java:167)
at
org.apache.acf.core.database.DBInterfaceDerby.performQuery(DBInterfaceDerby.java:727)
at
org.apache.acf.crawler.jobs.JobManager.makeJobStatus(JobManager.java:5611)
at
org.apache.acf.crawler.jobs.JobManager.getAllStatus(JobManager.java:5549)
at
org.apache.jsp.showjobstatus_jsp._jspService(showjobstatus_jsp.java:316)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be
obtained within the time requested
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
at
[jira] [Commented] (CONNECTORS-100) DB lock timeout, and/or indefinite or excessive database activity
[
https://issues.apache.org/jira/browse/CONNECTORS-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13133925#comment-13133925
]
Karl Wright commented on CONNECTORS-100:
The RSS connector's performance is because the time is close to a second per
fetch per document:
10-24-2011 05:30:50.423 fetch
http://localhost:8189/rss/gen.php?doc=4feed=782type=doc
200 46 843
I've created CONNECTORS-281 for this problem.
DB lock timeout, and/or indefinite or excessive database activity
-
Key: CONNECTORS-100
URL: https://issues.apache.org/jira/browse/CONNECTORS-100
Project: ManifoldCF
Issue Type: Bug
Components: Framework core
Affects Versions: ManifoldCF 0.1, ManifoldCF 0.2, ManifoldCF 0.3,
ManifoldCF 0.4
Environment: Running unmodified dist/example from trunk/ using the
default configuration.
Reporter: Andrzej Bialecki
Assignee: Karl Wright
Fix For: ManifoldCF 0.4
When a job is started and running (via crawler-ui) occasionally it's not
possible to display a list of running jobs. The problem persists even after
restarting ACF. The following exception is thrown in the console:
{code}
org.apache.acf.core.interfaces.ACFException: Database exception: Exception
doing query: A lock could not be obtained within the time requested
at
org.apache.acf.core.database.Database.executeViaThread(Database.java:421)
at
org.apache.acf.core.database.Database.executeUncachedQuery(Database.java:465)
at
org.apache.acf.core.database.Database$QueryCacheExecutor.create(Database.java:1072)
at
org.apache.acf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
at
org.apache.acf.core.database.Database.executeQuery(Database.java:167)
at
org.apache.acf.core.database.DBInterfaceDerby.performQuery(DBInterfaceDerby.java:727)
at
org.apache.acf.crawler.jobs.JobManager.makeJobStatus(JobManager.java:5611)
at
org.apache.acf.crawler.jobs.JobManager.getAllStatus(JobManager.java:5549)
at
org.apache.jsp.showjobstatus_jsp._jspService(showjobstatus_jsp.java:316)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be
obtained within the time requested
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
[jira] [Commented] (CONNECTORS-100) DB lock timeout, and/or indefinite or excessive database activity
[
https://issues.apache.org/jira/browse/CONNECTORS-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084147#comment-13084147
]
Karl Wright commented on CONNECTORS-100:
I haven't heard anything back from the Derby folks. I'm therefore going to
leave this ticket open. HSQLDB works better for the hopcount queries, although
it does not work well for the report queries. So I guess you can pick your
poison at the moment.
DB lock timeout, and/or indefinite or excessive database activity
-
Key: CONNECTORS-100
URL: https://issues.apache.org/jira/browse/CONNECTORS-100
Project: ManifoldCF
Issue Type: Bug
Components: Framework core
Environment: Running unmodified dist/example from trunk/ using the
default configuration.
Reporter: Andrzej Bialecki
Assignee: Karl Wright
When a job is started and running (via crawler-ui) occasionally it's not
possible to display a list of running jobs. The problem persists even after
restarting ACF. The following exception is thrown in the console:
{code}
org.apache.acf.core.interfaces.ACFException: Database exception: Exception
doing query: A lock could not be obtained within the time requested
at
org.apache.acf.core.database.Database.executeViaThread(Database.java:421)
at
org.apache.acf.core.database.Database.executeUncachedQuery(Database.java:465)
at
org.apache.acf.core.database.Database$QueryCacheExecutor.create(Database.java:1072)
at
org.apache.acf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
at
org.apache.acf.core.database.Database.executeQuery(Database.java:167)
at
org.apache.acf.core.database.DBInterfaceDerby.performQuery(DBInterfaceDerby.java:727)
at
org.apache.acf.crawler.jobs.JobManager.makeJobStatus(JobManager.java:5611)
at
org.apache.acf.crawler.jobs.JobManager.getAllStatus(JobManager.java:5549)
at
org.apache.jsp.showjobstatus_jsp._jspService(showjobstatus_jsp.java:316)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be
obtained within the time requested
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown
[jira] Commented: (CONNECTORS-100) DB lock timeout, and/or indefinite or excessive database activity
[
https://issues.apache.org/jira/browse/CONNECTORS-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13007773#comment-13007773
]
Karl Wright commented on CONNECTORS-100:
Confirmed. It's not an actual deadlock; it recovers after an extended period of
time. But it's not clear what query is slow, although this particular one
seems to be a possibility:
UPDATE hopcount SET deathmark='D',distance=-1 WHERE id IN(SELECT ownerid FROM
hopdeletedeps t0 WHERE ((t0.jobid=1300315252437 AND
t0.childidhash='0867FAD4FB2B46E04F2AFA9A1200D63266D48089')) AND EXISTS(SELECT
'x' FROM intrinsiclink t1 WHERE t1.linktype=t0.linktype AND t1.jobid=t0.jobid
AND t1.parentidhash=t0.parentidhash AND t1.childidhash=t0.childidhash))
Derby's diagnostics for query plan output seem inadequate to assess this in
real time, unfortunately.
DB lock timeout, and/or indefinite or excessive database activity
-
Key: CONNECTORS-100
URL: https://issues.apache.org/jira/browse/CONNECTORS-100
Project: ManifoldCF
Issue Type: Bug
Components: Framework core
Environment: Running unmodified dist/example from trunk/ using the
default configuration.
Reporter: Andrzej Bialecki
Assignee: Karl Wright
When a job is started and running (via crawler-ui) occasionally it's not
possible to display a list of running jobs. The problem persists even after
restarting ACF. The following exception is thrown in the console:
{code}
org.apache.acf.core.interfaces.ACFException: Database exception: Exception
doing query: A lock could not be obtained within the time requested
at
org.apache.acf.core.database.Database.executeViaThread(Database.java:421)
at
org.apache.acf.core.database.Database.executeUncachedQuery(Database.java:465)
at
org.apache.acf.core.database.Database$QueryCacheExecutor.create(Database.java:1072)
at
org.apache.acf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
at
org.apache.acf.core.database.Database.executeQuery(Database.java:167)
at
org.apache.acf.core.database.DBInterfaceDerby.performQuery(DBInterfaceDerby.java:727)
at
org.apache.acf.crawler.jobs.JobManager.makeJobStatus(JobManager.java:5611)
at
org.apache.acf.crawler.jobs.JobManager.getAllStatus(JobManager.java:5549)
at
org.apache.jsp.showjobstatus_jsp._jspService(showjobstatus_jsp.java:316)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be
obtained within the time requested
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at
