On Sun, Apr 27, 2014 at 10:24:34AM -0400, Peter Dietz wrote:
> Hi All,
>
> Calling all JDBC pooling experts. So, during the stress-testing of the
> REST-API many months ago, I've had jmeter hit REST full throttle for hours,
> fixed bugs in REST to support reliability and performance, and now it
> supports about a million requests per hour, depending on endpoint.
>
> However, I realized that I was doing something clever, or not-clever
> depending on perspective. REST doesn't participate in the database pool
> (getting a context, doing its thing, then returning the context at the end
> of the method). Instead I had stored a private static copy of the context,
> and never return it. Therefore, each endpoint holds its own connection,
> doesn't go through the overhead of getting a new context, and doesn't go
> through the overhead of closing the connection. This is quicker, but I've
> just discovered that it causes rest to not have real-time data, and it
> won't work when we add AuthN to rest.
>
> However, most importantly, I don't think that the DatabaseManager /
> DataSource / pool / context-stuff, properly supports this level of grabbing
> a new connection, and releasing the connection. My jmeter test hits 7
> endpoints, full throttle, and never stops.
>
> The current 4.0 version of REST uses "keep connection". It performs well,
> 0% error, 50-200 requests/per second, about a million per hour. However, by
> holding on to a static database connection, it can get
> less-than-real-time-data.
> if(context == null) {
> context = new org.dspace.core.Context();
> }
> ... do your thing
>
> If I change all the endpoints to use the pool, each request grabbing a new
> context, does its things, then closes the context, I get 80-100% error rate.
> The code for new context / close context looks like:
> context = new org.dspace.core.Context();
> ... do your thing
> context.complete();
>
>
> Errors are of the flavor:
>
> ERROR org.dspace.rest.CommunitiesResource @ This statement has been closed.
>
> ERROR org.dspace.rest.CommunitiesResource @ Cannot get a connection, pool
> error Timeout waiting for idle object
>
>
> So, to me, it looks like the database connection pool isn't working
> correctly, or isn't supposed to be used like this.Or your pool is too small for the load. How many concurrent connections are you running? OTOH some other part of the code could be leaking connections, or holding them for long periods. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient.
signature.asc
Description: Digital signature
------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs
_______________________________________________ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
