[
https://issues.apache.org/jira/browse/DERBY-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Knut Anders Hatlen updated DERBY-5632:
--------------------------------------
Attachment: experimental-v1.diff
I think there are two reasons why RAMAccessManager synchronizes on the
conglomerate cache instance whenever it accesses it:
1) Because it manually faults in missing items in the cache, and it needs to
ensure that no others fault it in between its calls to findCached() and
create().
2) Because conglomCacheUpdateEntry() implements a create-or-replace operation,
which is not provided by the CacheManager interface, and it needs to ensure no
others add an item with the same key between findCached() and create().
As mentioned in an earlier comment, I think (1) should be solved by
implementing CacheableConglomerate.setIdentity(), so that the cache manager
takes care of faulting in the conglomerate.
(2) might be solved by adding a create-or-replace operation to CacheManager
interface. However, I'm not sure it is needed. The conglomCacheUpdateEntry()
method is only called once; by RAMTransaction.addColumnToConglomerate(). That
method fetches a Conglomerate instance from the cache, modifies it, and
reinserts it into the cache. The instance that's reinserted into the cache is
the exact same instance that was fetched from the cache, so the call to
conglomCacheUpdateEntry() doesn't really update the conglomerate cache, it just
replaces an existing entry with itself.
It looks to me as if the conglomCacheUpdateEntry() can be removed, and that
will take care of (2).
I created an experimental patch, attached as experimental-v1.diff. It removes
conglomCacheUpdateEntry() as suggested. It also makes CacheableConglomerate
implement setIdentity() so that conglomCacheFind() doesn't need to fault in
conglomerates manually.
The patch is not ready for commit, as it doesn't pass all regression tests. But
it could be used for testing, if someone has a test environment where the
deadlock can be reliably reproduced.
There was only one failure in the regression tests. store/xaOffline1.sql had a
diff in one of the transaction table listings, where a transaction showed up in
the ACTIVE state whereas IDLE was expected.
This probably happens because the transaction used in the
CacheableConglomerate.setIdentity() method is not necessarily the same as the
one previously used by RAMAccessManager.conglomCacheFind().
The current implementation of setIdentity() in the patch just fetches the first
transaction it finds on the context stack. That seems to do the trick in most
cases, but it doesn't know whether conglomCacheFind() was called with a
top-level transaction or a nested transaction, as setIdentity() cannot access
conglomCacheFind()'s parameters. Maybe it can be solved by pushing some other
context type (with a reference to the correct tx) on the context stack before
accessing the conglomerate cache, and let setIdentity() check that instead?
> Logical deadlock happened when freezing/unfreezing the database
> ---------------------------------------------------------------
>
> Key: DERBY-5632
> URL: https://issues.apache.org/jira/browse/DERBY-5632
> Project: Derby
> Issue Type: Bug
> Components: Documentation, Services
> Affects Versions: 10.8.2.2
> Environment: Oracle M3000/Solaris 10
> Reporter: Brett Bergquist
> Labels: derby_triage10_10
> Attachments: experimental-v1.diff, stack.txt
>
>
> Tried to make a quick database backup by freezing the database, performing a
> ZFS snapshot, and then unfreezing the database. The database was frozen but
> then a connection to the database could not be established to unfreeze the
> database.
> Looking at the stack trace of the network server, , I see 3 threads that are
> trying to process a connection request. Each of these is waiting on:
> at
> org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(Unknown
> Source)
> - waiting to lock <0xfffffffd3a7fcc68> (a
> org.apache.derby.impl.services.cache.ConcurrentCache)
> That object is owned by:
> - locked <0xfffffffd3a7fcc68> (a
> org.apache.derby.impl.services.cache.ConcurrentCache)
> at
> org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.openGroupFetchScan(Unknown
> Source)
> at
> org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.updateIndexStatsMinion(Unknown
> Source)
> at
> org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.runExplicitly(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.AlterTableConstantAction.updateStatistics(Unknown
> Source)
> which itself is waiting for the object:
> at java.lang.Object.wait(Native Method)
> - waiting on <0xfffffffd3ac1d608> (a
> org.apache.derby.impl.store.raw.log.LogToFile)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
> - locked <0xfffffffd3ac1d608> (a
> org.apache.derby.impl.store.raw.log.LogToFile)
> at
> org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
> at
> org.apache.derby.impl.store.raw.data.BaseDataFileFactory.flush(Unknown Source)
> So basically what I think is happening is that the database is frozen, the
> statistics are being updated on another thread which has the
> "org.apache.derby.impl.services.cache.ConcurrentCache" locked and then waits
> for the LogToFile lock and the connecting threads are waiting to lock
> "org.apache.derby.impl.services.cache.ConcurrentCache" to connect and these
> are where the database is going to be unfrozen. Not a deadlock as far as
> the JVM is concerned but it will never leave this state either.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
