Re: OutOfMemoryErrors when testing Derby with DOTS

[Craig L Russell]

Hi John,

On Jan 31, 2006, at 9:14 AM, John Embretsen wrote:

Tuesday, January 31, 2006, 3:03:38 AM CET, Craig L Russell wrote:

Hi John,

On Jan 30, 2006, at 12:11 PM, John Embretsen wrote:

[snip old quotes]

Any reason why itemID is bound to the preparedStatement and not
itself passed as a parameter?

Generally, preparing statements like this (those that include
parameters) just before use has little benefit. The big win is to

the updateBidItemSQL to include the itemID as a ? parameter, then
pstmt.setInt(3, itemID) you will get the benefit of prepared
statements.

prepare the statement and then reuse it everywhere. So if you change

Craig

I am fully aware of this rather strange and inefficient use of
PreparedStatements in the DOTS test case. As I stated in my first E-
mail
in this thread:


 "sloppy code", or it may be intentional."

"I don't know the rationale behind doing it this way. It may be just

I was trying to be nice. I'll reiterate my point. If your use of
prepared statements is limited to the following pattern:

PreparedStatement ps = conn.prepare(savedSQLStatement + "" + itemID +
"");
ps.setInt(...);
ps.execute();

then you are not taking advantage of prepared statements. Your
prepared statement cache is not doing any good at all.

such use of PreparedStatements. I am not. I am testing Derby, and the

fixed.

I hope I have not given anyone the impression that I am an advocate for purpose of testing Derby is to find faults in Derby, so that they can be

I'm afraid I missed the opening shots of this discussion, and in particular where you said that you were using the referenced code as tests for the robustness of the various implementations. I would have put a slightly different slant on my comments.

Sure, changing or not running these tests will make the symptoms

guarantees that other users of Derby (directly or indirectly) will not
stumble across this or similar memory leaks in the future, when the
costs of experiencing and fixing such bugs may be a lot higher?

go away, but is that a good long term strategy for Derby? Do we have any

If this test code is representative of the actual application, then the application is in trouble and should be reimplemented in the jdbc area. It is a very well-understood requirement of well-behaved applications that result sets and prepared statements and connections be closed when they are no longer needed. And testing what happens to ill-behaved applications in various configurations doesn't seem to be a good use of anyone's energy. Long term, I don't believe that Derby or any other implementation should try to optimize for applications written without regard for good programming patterns. That said, OutOfMemoryError is unfortunate, but perhaps unavoidable. Does the test succeed, given enough memory? Does closing the result sets and prepared statements change the behavior? How can Derby know whether you intend to use the result set and prepared statement again, and you actually want to keep them open? Do you want Derby to internally close result sets and prepared statements that it guesses you no longer want? In a large system, wouldn't it be a bug in Derby if Derby closed result sets and prepared statements that the application still wanted?

This is the hammer that is making your head hurt. Before you can see
if aspirin helps, put down the hammer.

Personally, I would prefer a database with a strong enough helmet to
withstand such hammer hits. Someone else may find that hammer one
day, and hit you again ;)

Regards,

Craig

These particular hammers are in an area clearly marked "Danger: Use of these hammers may be injurious to your sanity". ;-) Seriously, if you are trying to evaluate different databases' behavior under a simulated application scenario, you should try to duplicate in your tests how the application actually behaves. And if the test is shown to have programming anomalies, and fixing the test informs better behavior in the application, then I'd say the test succeeded. After removing the anomalies in the test, you can see the underlying behavior of the system in various configurations, and have a better way to evaluate alternatives.

--
John

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:[EMAIL PROTECTED]
P.S. A good JDO? O, Gasp!

smime.p7s
Description: S/MIME cryptographic signature