-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Michael,
On 5/7/13 4:33 PM, Michael-O wrote:
> Am 2013-05-07 17:20, schrieb Christopher Schultz:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>>
>> Michael,
>>
>> On 5/7/13 6:43 AM, Michael-O wrote:
>>>> Von: "Mark Thomas" <ma...@apache.org> On 07/05/2013 10:25,
>>>> Michael-O wrote:
>>>>>
>>>>>> Von: "Mark Thomas" <ma...@apache.org> On 07/05/2013
>>>>>> 09:16, Michael-O wrote:
>>>>>>> Hi folks,
>>>>>>>
>>>>>>> I recently enabled a QueryTimeoutInterceptor with
>>>>>>> queryTimeout of 60 seconds in a JDBC Pool data source
>>>>>>> (7.0.37). When the app was shut down, Tomcat said: "The
>>>>>>> web application [/...] appears to have started a thread
>>>>>>> named [OracleTimeoutPollingThread] but has failed to
>>>>>>> stop it..."
>>>>>>>
>>>>>>> We are using Oracle 11.2g with 11.2.0.3 JDBC drivers.
>>>>>>> I have figured out that this thread is spawned by the
>>>>>>> driver itself. According to this Stackoverflow answer
>>>>>>> [1] this is a long-living thread, same says the JDBC
>>>>>>> FAQ [2] of Oracle.
>>>>>>>
>>>>>>> The thread seems to work like Pool's PoolCleaner
>>>>>>> thread. A few month a ago I reported the same issue
>>>>>>> with the PoolCleaner thread and Filip fixed the class
>>>>>>> loader orders.
>>>>>>>
>>>>>>> Can this be a false-positive by the memory leak
>>>>>>> detector since this thread lives only once in the
>>>>>>> entire VM?
>>>>>>
>>>>>> No. It is a memory leak and either or bug in your
>>>>>> application or in the JDBC driver.
>>>>>>
>>>>>> Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib
>>>>>> or WEB-INF/lib
>>>>>
>>>>> The driver is in the $CATALINA_HOME/lib only where
>>>>> $CATALINA_BASE != $CATALINA_HOME. This was done for a
>>>>> single webapp for testing purposes.
>>>>>
>>>>> Does this make a difference?
>>>>
>>>> The important thing is that it isn't in WEB-INF/lib.
>>>>
>>>>> How do you know that this is not a false-positive?
>>>>
>>>> Experience, a lot of research into memory leaks and I wrote
>>>> Tomcat's memory leak detection code.
>>>>
>>>>> If you really know for sure, I can open a service request
>>>>> with Oracle Support.
>>>>
>>>> Good luck with that.
>>>>
>>>> The problem is that when the driver creates the thread it
>>>> does so when the current class loader is the web application
>>>> class loader. That means that the Thread will be created with
>>>> a context class loader set to the web application class
>>>> loader. That will trigger a memory leak when the web
>>>> application is stopped because a reference is retained to the
>>>> web application's class loader.
>>>>
>>>> What the driver should do is, after it creates the thread,
>>>> set the thread's context class loader to the class loader
>>>> that loaded the driver.
>>>>
>>>> What you are seeing is a variation of the leak described on
>>>> page 15 of [1].
>>>
>>> After reading the slides and your explanation this makes
>>> sense. It's the same issue as "Pool cleaner thread should be
>>> created using the classloader that loaded the pool, not the
>>> context loader (fhanik)" fixed in 7.0.27.
>>>
>>> I will file SR and let you know.
>>
>> Note that you might be able to write your own code to mitigate
>> this problem, depending on exactly when that thread is created.
>> If the timeout thread isn't created until you actually try to
>> issue a query with a timeout, try something like this in a
>> ServletContextListener's contextInitialized method:
>>
>> // NOTE: No resource management is being done in this example
>>
>> // Get the current ClassLoader -- should be WebappClassLoader
>> ClassLoader cl = Thread.currentThread().getContextClassLoader();
>>
>> // WebappClassLoader.getParent should be "common" loader
>> Thread.currentThread().setContextClassLoader(cl.getParent());
>>
>> try { Connection conn = ...; // However you get a connection
>> Statement s = conn.createStatement(); s.setQueryTimeout(5000); //
>> Doesn't really matter what TO is s.execute("SELECT 1 FROM dual",
>> ); } .... finally { // Pop back to the original ClassLoader
>> Thread.currentThread().setContextClassLoader(cl); }
>>
>> This is the exact technique that Tomcat's
>> JreMemoryLeakPreventionListener uses to prevent ClassLoader
>> leaks.
>
> Yes, this looks like the way JDBC Pool does with the Pool Cleaner.
> I would go for this as a last resort.
>
>> A couple of notes:
>>
>> 1. This won't work under a SecurityManager. If you need to
>> operate under a SecurityManager, have a look at the
>> JreMemoryLeakPreventionListener code and adapt it to the above.
>>
>> 2. If the Oracle driver launches the thread when the DataSource
>> is created, it might happen too early for a
>> ServletContextListener to intervene. In that case, simply modify
>> the JreMEmoryLeakPreventionListener code directly. Patches are
>> always welcome.
>
> Not, it is not. I have attached VisualVM to Tomcat VM and have seen
> that the thread is forked when the first query is executed.
That's good news: it means that if you need it fixed *right away*, you
don't need to hack-and-recompile Tomcat: you can just do it in your
own ServletContextListener.
>> 3. If Oracle fixes this bug, Tomcat should not prevent against
>> it. If Oracle refuses to acknowledge/fix/etc. this bug, then it
>> may make sense to include such code in
>> JreMemoryLeakPreventionListener, but it should probably be done
>> in such a way that a) it's not enabled by default and b) the
>> query used for triggering the Thread to be created is
>> user-selectable with maybe a reasonable default (like "SELECT 1
>> FROM dual", as that tends to be valid in most RDBMSs).
>
> I am already in contact with an Oracle engineer who has received a
> demo WAR file to reproduce this issue. If Oracle won't, maybe some
> generic approach would be advisable but in the in
> JreMemoryLeakPreventionListener but rather in Tomcat JDBC Pool
> IMHO.
I think this might be a rather esoteric thing to exercise, but one
could certainly add it as an interceptor that is not enabled by default.
> I do not know how other RDBMS vendors implement the timeout
> function.
MySQL has taken some time to fix similar/related bugs, but is
(eventually) willing to fix them:
http://bugs.mysql.com/bug.php?id=65909
http://bugs.mysql.com/bug.php?id=36565
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRiXrJAAoJEBzwKT+lPKRYOqIP/1VeVgEvA82+uAn6x9kSKoQK
HN5zOIUwUG/Mr5zWJjR7XeZoOSKaiYHb6xZOjM9zG7hROrHdfrvoj6YLO+GNIe73
wPw9Tx9HGww6NskBYReG/TtS1DKLODsxEd5xkngckDH+xrTr3P0lvhSSpg5pS5c6
gmMqw/PMP6fjvN2JeOtETShxzVYXVX3fh9tYLATPiJuZnMT1v3p0Jo7AZHX7ucRr
LnoJm/+VbszQkXq0Bve+FRyE7y4yXgc7J90/FOiJ/B5DJ7a4LQB8jIp88DGFBEKv
+s2Yrlf2R39mAtGA1Vj6KEVxvaEW9zIR0wPDuriR6zdxys3kkkYyN51TTkUGOsHA
FyFtAlJ0XqAe5oHxeNzW+P+4cn2wzBAz48A6Snhtf8aZoCzRQqJpz9GZfbtF/3YM
XLXD9zxpq4r6m61eT8Ai8wh0tz0BDCPDshjh7zD8F99Ov+NGtjI101+Fkkz2J+tM
ZDKKWrhfKPXjN47BCYTCw7bM2J9HexJc/Xp9mSr7uI7JaHPcoFPIyH7PJkJLOclZ
n4b2SZphYO3WEmyToIWimTupqJIBMYt+jrCC35qOzrSLXUVknA4oWI2IwHjjxNue
eoO8Ao6kMlv6+SGKoZGdW+nZCG40tZ7KVPOTPsClZwes/eeOCQrZHnWJCZqZ95qO
l9oE3KNfNv1Hu1bjFcyj
=oD8m
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
