Hi Shapira y and tomcat users,
#You're putting too many things in one message. ;)
Can't help that...;), in fact I would like to add some more things here
:).
#So with your configuration, you'd always have between 30*12 = 360 and
#100*12 = 1200 open database connections. Is this really what you want?
#If your DB can't handle 1000 connections, much less 1200, configure
your
#pools so that they don't reach this max.
Yeah right but there is a catch here. Because we are using the dynamic
scheme the connections can go beyond the 100 connections which is the
max connections. Seems, the problem comes up when the number of
connections cross 250 on any individual tomcat. What we want is whenever
there is this surge of load the connection pool should be able to
request the same. I don't have pointers which says that 200-250
connections size of oracle connection pool is not supported by
classes12.jar and only have observation.
#Not sure is right. Threads could be waiting on a lot of things, and
#it's not always bad to wait (e.g. idle HTTP processor threads). Which
#threads were waiting and was it for a common monitor address?
Why I thought that it was waiting for closing the logical connections
was because the thread dump had some pointers and this time I am
enclosing those snapshots.
"Thread-360" prio=5 tid=0x00ac4f50 nid=0xc99 waiting for monitor entry
[b1080000..b1081998]
at
oracle.jdbc.pool.OracleConnectionEventListener.connectionClosed(OracleCo
nnectionEventListener.java:125)
- waiting to lock <0xd4d2c820> (a
oracle.jdbc.pool.OracleConnectionEventListener)
at
oracle.jdbc.pool.OraclePooledConnection.callListener(OraclePooledConnect
ion.java:482)
at
oracle.jdbc.pool.OraclePooledConnection.logicalClose(OraclePooledConnect
ion.java:445)
- locked <0xcc4f5700> (a
oracle.jdbc.pool.OraclePooledConnection)
at
oracle.jdbc.driver.OracleConnection.logicalClose(OracleConnection.java:2
900)
- locked <0xcc4f57c0> (a oracle.jdbc.driver.OracleConnection)
at
oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:1418)
- locked <0xcc4f57c0> (a oracle.jdbc.driver.OracleConnection)
"Thread-360" prio=5 tid=0x00ac4f50 nid=0xc99 waiting for monitor entry
[b1080000..b1081998]
at
oracle.jdbc.pool.OracleConnectionEventListener.connectionClosed(OracleCo
nnectionEventListener.java:125)
- waiting to lock <0xd4d2c820> (a
oracle.jdbc.pool.OracleConnectionEventListener)
at
oracle.jdbc.pool.OraclePooledConnection.callListener(OraclePooledConnect
ion.java:482)
at
oracle.jdbc.pool.OraclePooledConnection.logicalClose(OraclePooledConnect
ion.java:445)
- locked <0xcc4f5700> (a
oracle.jdbc.pool.OraclePooledConnection)
at
oracle.jdbc.driver.OracleConnection.logicalClose(OracleConnection.java:2
900)
- locked <0xcc4f57c0> (a oracle.jdbc.driver.OracleConnection)
at
oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:1418)
- locked <0xcc4f57c0> (a oracle.jdbc.driver.OracleConnection)
"Thread-358" prio=5 tid=0x00ac37a0 nid=0xc97 waiting for monitor entry
[b1280000..b1281998]
at
oracle.jdbc.pool.OracleConnectionEventListener.connectionClosed(OracleCo
nnectionEventListener.java:125)
- waiting to lock <0xd4d2c820> (a
oracle.jdbc.pool.OracleConnectionEventListener)
at
oracle.jdbc.pool.OraclePooledConnection.callListener(OraclePooledConnect
ion.java:482)
at
oracle.jdbc.pool.OraclePooledConnection.logicalClose(OraclePooledConnect
ion.java:445)
- locked <0xc51d5eb8> (a
oracle.jdbc.pool.OraclePooledConnection)
at
oracle.jdbc.driver.OracleConnection.logicalClose(OracleConnection.java:2
900)
- locked <0xc51d5f78> (a oracle.jdbc.driver.OracleConnection)
at
oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:1418)
- locked <0xc51d5f78> (a oracle.jdbc.driver.OracleConnection)
"Thread-357" prio=5 tid=0x00ac30c0 nid=0xc96 runnable
[b1380000..b1381998]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at
java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
at
java.io.BufferedInputStream.read(BufferedInputStream.java:277)
- locked <0xc73bdcf0> (a java.io.BufferedInputStream)
"Thread-356" prio=5 tid=0x00ac2a38 nid=0xc95 waiting for monitor entry
[b1480000..b1481998]
at
oracle.jdbc.pool.OracleConnectionEventListener.connectionClosed(OracleCo
nnectionEventListener.java:125)
- waiting to lock <0xd4d2c820> (a
oracle.jdbc.pool.OracleConnectionEventListener)
at
oracle.jdbc.pool.OraclePooledConnection.callListener(OraclePooledConnect
ion.java:482)
at
oracle.jdbc.pool.OraclePooledConnection.logicalClose(OraclePooledConnect
ion.java:445)
- locked <0xc9ffd1c8> (a
oracle.jdbc.pool.OraclePooledConnection)
at
oracle.jdbc.driver.OracleConnection.logicalClose(OracleConnection.java:2
900)
- locked <0xc9ffd288> (a oracle.jdbc.driver.OracleConnection)
at
oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:1418)
- locked <0xc9ffd288> (a oracle.jdbc.driver.OracleConnection)
I have given this a close look this time and seems that some of these
threads are waiting to accquire a lock on
oracle.jdbc.pool.OracleConnectionEventListener object ( <0xd4d2c820> -
not sure if this the object ID or objects's memory address ). It seems
that the reaper thread (not sure !!! may be have to decompile the oracle
driver's code) creates this
oracle.jdbc.pool.OracleConnectionEventListener object and because its
dead due to some reasons all the threads continue in their wait state to
accquire the lock on this. If this is a known bug then VOILA!!!.
#What is killing the oracle reaper?
If I had known this, I would be a happy man... :-< Right now I am
investing my energies to prove a lot of things and this being one of
them.
#What is the date of the drivers in classes12.zip? And you've renamed
it
#classes12.jar, right? There are better drivers that come with Oracle
9i
#(v9.2.0.1), the file is called ojdbc14.jar, and it's
#backwards-compatible for Oracle 8i. You should try these.
Absolutely, renamed that. Yeah we are in the process to see all the
areas of improvement. Thanks for the suggestion, I have taken a note.
#Then fix the memory leaks.
Not an easy fix, there are many java classes and this will take a lot of
time. We want to keep this thing as a long term plan under application
improvement heading.
#The JVM will use as much memory as you let it. Monotonic increase is
to
#be expected when you look at it via the top command. What you're more
#interested in is the %free of the heap. See how well your apps run if
#you reduce max memory from 1GB to say 256MB.
No, this will have negative impact on the application because in the
past its was observed under such conditions of low memory, the GC
activity stops (increased pause time) the application and there is no
other way but to kill the tomcat. I don't think this is a viable option
and its risky too. I don't know the impact. Clients would kill us if
something goes wrong. This is one of the biggest challenges I am facing
in this project.
#Attachments didn't come through, and a GC log wouldn't be very
#meaningful to us anyways.
I am sorry but the point I was making here was during the course of day,
the heap size keeps on increasing but don't know if this is due to
memory leak or JVM getting vey greedy these days. Eats up all it gets
;-)
Two more problems:
1 - We are serving some downloads from the site too. There are 2 types
of downloads - Authorised and Non-authorised. Authorised content have to
be downloaded from tomcat and non-auth from apache through a rewrite
rule. Problem is that there is high thread count when there is
authorised content is downloaded. These downloads are of the size of 1-5
MB. Has these issues been solved in tomcat 4.0 or higher? I will check
it otherwise. (Hell lot of things to do!!!)
2 - 12 tomcat having only one shared code base which is mounted on an
NFS (this is too much, one might say). I just want to know if all the
error logs (Error logger thread exact) are also written to the same NFS
part this will call for high I/O contention (right???) because the
writting is synczd. Secondly, what are the affects what so ever when the
classes are loaded during runtime? (again I/O)
As and when I have more things revealed or remember will through in
here...I forget as I write...memory leak else where too...
I am really impressed that you have given me some response...and expect
more...haven't heard much from anybody else from the group.
Last but not least needed to know, if we can have some support contract
from apache group to support our application. Is that possible?
Thanks for the patience. (It takes time to compose this mail...I need
patience too)
~Arnab
This e-mail, including any attachments, is a confidential business
communication, and may contain information that is confidential,
proprietary and/or privileged. This e-mail is intended only for the
individual(s) to whom it is addressed, and may not be saved, copied,
printed, disclosed or used by anyone else. If you are not the(an)
intended recipient, please immediately delete this e-mail from your
computer system and notify the sender. Thank you.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
