![]() |
|
|
|
|
Issue Type:
|
![Bug]() Bug
|
|
Affects Versions:
|
1.8.0 |
|
Assignee:
|
Unassigned |
|
Components:
|
OAI-PMH |
|
Created:
|
23/Sep/13 3:25 PM
|
|
Description:
|
On http://dspace.mit.edu after upgrading from 1.6 to 1.8, we started seeing serious failures in our OAI gateway. Specifically, after a certain amount of time after the last restart, we would start getting HTTP 500 errors, and exceptions complaining that it could not retrieve a connection from the pool.
We have a high level of traffic on DSpace@MIT, and we determined that it was concurrent traffic on the OAI gateway that was leading to these connections bleeding out of the OAI connection pool. Upon investigating what seemed like a threading issue, we saw two distinct problems:
1.) Upon an unhandled exceptions, connections were not being returned properly.
2.) We were seeing collisions in the METS building code that builds the OAI output (the cause of the above exceptions), specifically within the Harvard classes that are not local to the DSpace code-base.
Our fixes / workaround for these issues are as follows:
1.) We moved the context.complete() call to a finally block within the METS crosswalk to properly clean up in the case of an exception.
Index: dspace-api/src/main/java/org/dspace/content/crosswalk/METSDisseminationCrosswalk.java
===================================================================
--- dspace-api/src/main/java/org/dspace/content/crosswalk/METSDisseminationCrosswalk.java (revision 2452)
+++ dspace-api/src/main/java/org/dspace/content/crosswalk/METSDisseminationCrosswalk.java (revision 2459)
@@ -107,6 +107,7 @@
throw new CrosswalkInternalException("Cannot find a disseminate plugin for package=" + METS_PACKAGER_PLUGIN);
}
+ Context context = null;
try
{
// Set the manifestOnly=true param so we just get METS document (and not content files, etc)
@@ -119,9 +120,8 @@
tempFile.deleteOnExit();
// Disseminate METS to temp file
- Context context = new Context();
+ context = new Context();
dip.disseminate(context, dso, pparams, tempFile);
- context.complete();
try
{
@@ -139,6 +139,12 @@
{
throw new CrosswalkInternalException("Failed making METS manifest in packager (see wrapped error message for more details) ",pe);
}
+ finally {
+ if(null != context)
+ {
+ context.complete();
+ }
+ }
}
2.) Since the METS disseminator was being cached in the PluginManager, its local reference to the OAI list used to build METS output was being reused, causing the collisions we were seeing. Since we were not currently managing the Haravard OAI code, instead of trying to fix the concurrency issues within, we set the METS disseminator class that was being cached in the PluginManager to not be cached, via a relatively undocumented configuration parameter that allows one to exclude plugins from being cached:
plugin.reusable.org.dspace.content.packager.DSpaceMETSDisseminator = false
These changes prevented the collisions, and also protected against any connection pool leaks that would occur if we did get unexpected exceptions.
|
|
Project:
|
DSpace
|
|
Labels:
|
oai
connectionpool
leak
|
|
Priority:
|
![Minor]() Minor
|
|
Reporter:
|
Sands Fish
|
|
|
|
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
|
------------------------------------------------------------------------------
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
_______________________________________________
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
