Issue with SYSCS_UTIL.SYSCS_COMPRESS_ TABLE
Hello, Our application using Derby 10.4 (in embedded mode) has a background process that runs the SYSCS_UTIL.SYSCS_COMPRESS_TABLE procedure periodically. We have been reported the following error while this procedure is being executed and a SELECT statement came in at the same time: com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. com.linoma.dpa.runtime.JobFailedException: com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. at com.linoma.dpa.runtime.Job.checkLogin(Unknown Source) at com.linoma.dpa.runtime.Job.run(Unknown Source) at com.linoma.dpa.runtime.Runtime.executeProject(Unknown Source) at com.linoma.dpa.runtime.Runtime.executeProject(Unknown Source) at com.linoma.dpa.j2ee.RunProjectCommandServlet.doPost(Unknown Source) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.linoma.dpa.security.SecurityFilter.doFilter(Unknown Source) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:810) Caused by: com.linoma.dpa.security.LoginException: com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. at com.linoma.dpa.security.LoginUtilities.getUserDetails(Unknown Source) ... 22 more Caused by: com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. at com.linoma.dpa.dao.rdbms.UserDAOImpl.getConsolidatedRoles(Unknown Source) at com.linoma.dpa.dao.rdbms.UserDAOImpl.findUser(Unknown Source) ... 23 more Caused by: java.sql.SQLException: The conglomerate (71,409) requested does not exist. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeQuery(Unknown Source) at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93) ... 25 more Caused by: ERROR XSAI2: The conglomerate (71,409) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.btree.index.B2IFactory.readConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.store.access.BackingStoreHashTableFromScan.init(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.createBackingStoreHashtableFromScan(Unknown Source) at org.apache.derby.impl.sql.execute.HashScanResultSet.openCore(Unknown Source)
Re: Issue with SYSCS_UTIL.SYSCS_COMPRESS_ TABLE
Sai Pullabhotla wrote: Hello, Hello and thank you for reporting this issue. Our application using Derby 10.4 (in embedded mode) has a background process that runs the SYSCS_UTIL.SYSCS_COMPRESS_TABLE procedure periodically. We have been reported the following error while this procedure is being executed and a SELECT statement came in at the same time: com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. Mike just fixed a corruption issue related to inplace compress,https://issues.apache.org/jira/browse/DERBY-4239 but my understanding is that it should not be an issue with offline compress and also I would expect your database to be unbootable even after you completely shutdown your application and Derby.Can you 1) Post the full nested stack trace from the derby.log. 2) Run a consistency check on your database. http://wiki.apache.org/db-derby/DatabaseConsistencyCheck 4) Tell us the isolation level of your select statement. 3) Modify the reproduction attached to DERBY-4239 https://issues.apache.org/jira/secure/attachment/12408948/reproBackgroundCheckpoint.zip to use SYCS_UTIL.SYSCS_COMPRESS_TABLE and see if you can reproduce your issue. You might need to modify it to add in a select. Thanks Kathey
Re: Issue with SYSCS_UTIL.SYSCS_COMPRESS_ TABLE
Kathey, Thanks for replying. This error occurred on a production system at a customer's site and the stack trace I posted is what I've available. Since the customer restarted the app, by the time we got the derby.log, it was empty (I guess the log is overwritten everytime?). Here is the sequence on what happened when the failure occurred: 6/3/09 12:04:58 AMINFO Preparing to backup the database 6/3/09 12:05:14 AMINFO Database was successfully backed up to /linoma/goanywhere/userdata/database/backups/2009-06-03.zip. 6/3/09 12:05:14 AMINFO Backup operation took 16 seconds. 6/3/09 12:05:14 AMINFO Compressing and performance tuning the database tables... 6/3/09 12:05:39 AMINFO Job 1243782054134 started for project '/PTP Tax Upload' 6/3/09 12:05:40 AMERROR Job 1243782054134 completed with an error com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. [STACK TRACE OMMITTED AS IT WAS POSTED IN THE ORIGINAL POST] 6/3/09 12:05:43 AMINFO Database compression completed successfully. Operation took 28 seconds. 6/3/09 12:05:43 AMINFO Verifying the database consistency... 6/3/09 12:05:49 AMINFO Database consistency check completed successfully. Operation took 5 seconds. 6/3/09 12:10:39 AMINFO Job 1243782054135 started for project '/PTP Tax Upload' 6/3/09 12:10:39 AMERROR Job 1243782054135 completed with an error com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. [SEVERAL JOBS FAILED WITH THE SAME EXCEPTION UNTIL THE SYSTEM WAS RESTARTED] As you can see, we first did a backup using the stored procedure - SYSCS_UTIL.SYSCS_BACKUP_DATABASE. Then we started the compression of all the tables... while the tables are being compressed, a SELECT statement was executed because of a job submission. This SELECT errored out. Then the compression completed normally. Then we performed a database consistency check using the following statement: SELECT schemaname || '.' || tablename as TableName, SYSCS_UTIL.SYSCS_CHECK_TABLE(schemaname, tablename) AS OK FROM sys.sysschemas s, sys.systables t WHERE s.schemaid = t.schemaid and t.tabletype = 'T' The consistency check succeeded with no issues. All subsequent queries failed with the same error until the app was restarted. The isolation level is READ_COMMITTED (2). I will try to see if I can reproduce the error here and send you any additional logs. Sai Pullabhotla www.jMethods.com On Fri, Jun 5, 2009 at 1:25 PM, Kathey Marsden kmarsdende...@sbcglobal.netwrote: Sai Pullabhotla wrote: Hello, Hello and thank you for reporting this issue. Our application using Derby 10.4 (in embedded mode) has a background process that runs the SYSCS_UTIL.SYSCS_COMPRESS_TABLE procedure periodically. We have been reported the following error while this procedure is being executed and a SELECT statement came in at the same time: com.linoma.dpa.dao.DAOException: The conglomerate (71,409) requested does not exist. Mike just fixed a corruption issue related to inplace compress, https://issues.apache.org/jira/browse/DERBY-4239 but my understanding is that it should not be an issue with offline compress and also I would expect your database to be unbootable even after you completely shutdown your application and Derby.Can you 1) Post the full nested stack trace from the derby.log. 2) Run a consistency check on your database. http://wiki.apache.org/db-derby/DatabaseConsistencyCheck 4) Tell us the isolation level of your select statement. 3) Modify the reproduction attached to DERBY-4239 https://issues.apache.org/jira/secure/attachment/12408948/reproBackgroundCheckpoint.zip to use SYCS_UTIL.SYSCS_COMPRESS_TABLE and see if you can reproduce your issue. You might need to modify it to add in a select. Thanks Kathey
Re: Issue with SYSCS_UTIL.SYSCS_COMPRESS_ TABLE
Sai Pullabhotla wrote: Since the customer restarted the app, by the time we got the derby.log, it was empty (I guess the log is overwritten everytime?). It does unless unless you set this property: http://db.apache.org/derby/docs/10.5/ref/rrefproper13217.html#rrefproper13217 I will try to see if I can reproduce the error here and send you any additional logs. It looks like you have a good understanding of what was going on at the time, so hopefully you can come up with a reproduction. Even if you cannot, post what you have tried and perhaps someone can offer some suggestions or tweak it to pop the bug. Kathey
Multiple instances of an application using same embedded database?
Hi all- I am writing an application that uses embedded Derby for holding data that the application collects/creates. It works great, I really like Derby. But, there's a problem: multiple application instances. The application can be launched multiple times and each instance can potentially create and retrieve data that would be stored/queried from Derby. The problem is that the second instance fails to run correctly because the database is locked by the first instance of the application. I can't create a Derby server, the application must be zero-setup and zero-config; the application handles initial database creation, making the tables, etc. I've looked and there doesn't seem to be any info on this. Is this even possible? If not, might anyone know what I might be able to use? The original version of the program used SQLite, which keeps the database in the single file. This was nice, but there isn't a pure Java way to get at a SQLite db, and besides, I like the additional functionality Derby gives me. Thanks for any info, Tacho
Re: Multiple instances of an application using same embedded database?
Have a look at the archives of this list. It has been discussed already that you can use server mode similarly to embedded. Your application may try first use existing database as client and if fails to connect it may start server. Actually your second application instance would benefit from accessing as client because of derby server cache. On Fri, Jun 5, 2009 at 10:51 PM, Ron Olson tachokni...@gmail.com wrote: Hi all- I am writing an application that uses embedded Derby for holding data that the application collects/creates. It works great, I really like Derby. But, there's a problem: multiple application instances. The application can be launched multiple times and each instance can potentially create and retrieve data that would be stored/queried from Derby. The problem is that the second instance fails to run correctly because the database is locked by the first instance of the application. I can't create a Derby server, the application must be zero-setup and zero-config; the application handles initial database creation, making the tables, etc. I've looked and there doesn't seem to be any info on this. Is this even possible? If not, might anyone know what I might be able to use? The original version of the program used SQLite, which keeps the database in the single file. This was nice, but there isn't a pure Java way to get at a SQLite db, and besides, I like the additional functionality Derby gives me. Thanks for any info, Tacho
Re: Multiple instances of an application using same embedded database?
On Fri, 2009-06-05 at 22:58 +0200, Peter Ondruška wrote: Have a look at the archives of this list. It has been discussed already that you can use server mode similarly to embedded. Your application may try first use existing database as client and if fails to connect it may start server. Hi, I guess you could have the application try to connect using the embedded driver first, and if that fails it could assume another application instance has already booted the Derby database. In that case, the second instance could fall back to connect using the client driver (localhost, custom port maybe). There are multiple issues with this; - various race conditions to handle (pausing, retry) - the first instance to boot the database must start the server (doing this programatically is easy) - failover and retry logic for handling the case where the first application shuts down before the second instance - even though we try to keep the embedded and client driver fully compatible, there may be small differences between them - you could theoretically make your application instances talk to each other, such that you can request that a Derby server is started by the first instance Depending on the usage pattern of the application, this may get complicated... Is it a possibility to boot/create different databases, or does each instance require access to the same data (i.e. from earlier sessions)? Cheers, -- Kristian Actually your second application instance would benefit from accessing as client because of derby server cache. On Fri, Jun 5, 2009 at 10:51 PM, Ron Olson tachokni...@gmail.com wrote: Hi all- I am writing an application that uses embedded Derby for holding data that the application collects/creates. It works great, I really like Derby. But, there's a problem: multiple application instances. The application can be launched multiple times and each instance can potentially create and retrieve data that would be stored/queried from Derby. The problem is that the second instance fails to run correctly because the database is locked by the first instance of the application. I can't create a Derby server, the application must be zero-setup and zero-config; the application handles initial database creation, making the tables, etc. I've looked and there doesn't seem to be any info on this. Is this even possible? If not, might anyone know what I might be able to use? The original version of the program used SQLite, which keeps the database in the single file. This was nice, but there isn't a pure Java way to get at a SQLite db, and besides, I like the additional functionality Derby gives me. Thanks for any info, Tacho
