On Mon, 24 Mar 2003, Danny Angus wrote:
Date: Mon, 24 Mar 2003 22:28:29 -0000 From: Danny Angus <[EMAIL PROTECTED]> Reply-To: Jakarta Commons Developers List <[EMAIL PROTECTED]> To: Jakarta Commons Developers List <[EMAIL PROTECTED]> Subject: RE: [DBCP] Please let us start to improve the connection pool
Craig,
IMHO, any application that depends on the connection pool for recovering abandoned connections (whether or not it recycles them) is broken. Far better is to focus your energy on avoiding all the cases where you grab a connection from the pool but fail to return it for some reason.
One simple way to do this is to encapsulate your JDBC-using code in a try/catch/finally block, something like this:
I haven't used DBCP for anthing yet, though we're proposing to use it for James in place of our homespun pool.
However from what I understand of this discussion there are two things going on here;
Thing one, I agree with you, code failing to return connections to the pool should lead to failure, and the sooner this can be done the better for identifying such broken code.
Thing two, DBCP will apparently make a value judgement about an assigned connection, and is capable of recycling it with no notification to the code which has checked it out.
will --> can be configured to
In my opinion thing two is wrong or incomplete as it creates a situation where potential failure is built in, difficult to reproduce and difficult identify the cause of.
There is a lot of flexibility in how you configure DBCP -- most of it inherited from the commons-pool capabilities that lie underneath. In all cases, the use of these features is configurable.
* Test On Borrow -- The pool can perform some tests on an object instance that the pool is about to return to the application that asked for it.
* Test On Return -- The pool can perform the same sorts of tests, but this time when the application returns the instance to the pool.
* Test While Idle -- The pool can perform checks on connections that are in the pool (not currently allocated to an application) to make sure that it is still valid.
DBCP (in particular, the BasicDataSource class found there) implements flags to enable each of these behaviors individually, along with a validation query that is used to perform the test. In addition, there are tuning parameters to control things like how often the "test when idle" checks are performed, and how many connections are tested each time.
In the case of JDBC connection pooling may be reasonable to want to keep a connection even when it is idle, because connections can aquire state which is expensive to reproduce. Is it not, then, unresonable to allow the pool to silently and forcably recycle apparently idle but valuable connections?
Keeping physical JDBC connections open between uses by the app, and allowing those connections to be shared, is the whole point of DBCP :-).
But I think we might be using the term "recycle" differently.
By "recycle", do you mean "if a connection has been setting in the pool for a long time and is not allocated to an application, so we can close it now"? If so, that behavior is already configurable -- just set the maxIdle property to zero and no idle connection will ever get harvested. There's no application to notify in this case, because the instance being harvested wasn't allocated to an application at the time.
Or, by "recycle", do you mean "if a connection has been allocated to an application but not returned for a long time, the pool is allowed to grab it back again -- but you want to notify the application first". The "grab it back" behavior is already configurable (and not enabled by default) -- and it's this functionality that I object to having at all. If there was no "grab it back" we wouldn't have to worry about notifying anyone that it was about to happen :-).
My solution would either be to make it possible to turn off the forcable recycling of connections, or to make the pool capable of notifying code that its connection has been recycled.
You can already turn off the forcible recycling -- in fact, you explicitly have to turn it *on* (i.e., for BasicDataSource, you have to explicitly set the "removeAbandoned" property and related values to enable it).
But, I contend that no one should ever do that :-).
Is that reasonable?
If we're talking about the second use of "recycled" above, IMHO, I think adding support for recovering abandoned connections at all was a mistake. Doing anything to make it work "better" (knowing all the while that it cannot be made "perfect") simply perpetuates the mistake. I'd much rather see this whole area of functionality deprecated, rather than continuing to mislead people into believing that its OK to depend on something that cannot ever work reliably 100% of the time.
But that's just my opinion.
I think Craig is correct.
If code is not closing connections (and thereby not returning them to the pool), it is not the responsibility of the pool to try and recover them. Applications would suffer the same problem if they didn't use pooling. Pooling is there to try and make applications faster and more efficient, not to cover up bugs.
If our code is not closing connections or other resources we want to find out about this as soon as possible during development. If 'abandoned' connections are being recovered by the pool, that obscures the fact. It could lead to the problem causing intermittent connection problems in production, exactly where you don't want it.
DBCP (& Pool) should help us see any database connection problems (which it is presently hiding) such as problems creating a new db connection, evicting an idle connection or a failed validation.
I think effort should be put into this area by adding commons-logging to DBCP and Pool. In particular, DelegateConnection.passivate() and DelegateStatement.passivate() would be good candidates as they close open Statements and ResultSets that the application should have closed.
To help locate code which has failed to close a connection, it would be nice to have a list of 'active' connections and the length of time they have been active. As I funnel all of my connection requests through a static method in my code I have a central place where I can log my connection requests. I will probably add some code to record the Connection.hashCode() and the time to a hashmap so I can generate a report on the number/time connections are left open. I will also use a static method to close my connections so I can remove from the hashmap and log the close(). (I wonder if DBCP/Pool should do such as thing? It guess it is really only useful for debugging or possibly displaying statistics on pool usage eg average time object is used)
Jason Lea
d.
Craig
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
-- Jason Lea Email: [EMAIL PROTECTED] Phone/Fax: +64 3 381 2907 Mobile: +64 21 040 2708 Address: 9a Tabart Street, Christchurch, New Zealand
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
