Hi,
We have a production environment with jboss 4.2.3. It has been working
successfully until now, when we are experiencing network issues. TCP
connections are broken very often. Also we noticed that a great number of tcp
connections in a ESTABLISHED status were beginning to increase out of control.
We have a pool of 9 max connections and we had 3 hundred of ESTABLISHED
connections and rising. So we had a look at the code and noticed a bug in
connection validation, in
org.jboss.resource.connectionmanager.InternalManagedConnectionPool class
validateConnections method.
| boolean destroyed = false;
|
| try
| {
|
| while (true)
| {
|
| ConnectionListener cl = null;
|
| synchronized (cls)
| {
| if (cls.size() == 0)
| {
| break;
| }
|
| cl = removeForFrequencyCheck();
|
| }
|
| if (cl == null)
| {
| break;
| }
|
| try
| {
|
| Set candidateSet =
Collections.singleton(cl.getManagedConnection());
|
| if (mcf instanceof ValidatingManagedConnectionFactory)
| {
| ValidatingManagedConnectionFactory vcf =
(ValidatingManagedConnectionFactory) mcf;
| candidateSet = vcf.getInvalidConnections(candidateSet);
|
| if (candidateSet != null && candidateSet.size() > 0)
| {
|
| if (cl.getState() != ConnectionListener.DESTROY)
| {
| doDestroy(cl);
| destroyed = true;
| }
| }
|
| }
| else
| {
| log.warn("warning: background validation was specified
with a non compliant ManagedConnectionFactory interface.");
| }
|
| }
| finally
| {
| if(!destroyed)
| {
| synchronized (cls)
| {
| returnForFrequencyCheck(cl);
| }
| }
|
| }
|
| }
|
| }
| finally
| {
| permits.release();
|
| if (destroyed && shutdown.get() == false && poolParams.minSize
> 0)
| {
| PoolFiller.fillPool(this);
| }
|
| }
|
What happens when you have a set of 3 connections in the pool, and when you run
this validation code, the fist one is not valid and the following 2 are valid?
I will tell you. Those 2 valid connections remain established but they don't
return to the pool, resulting that you have 2 established connections out of
control. Besides, counters don't get updated and the resource adapter turns
into an inconsistent status. All of this is because you don't update destroyed
variable to false on every iteration. I think it is not necesary to explain
this more in detail...
So I have developed the following simple solution to fix this issue:
| boolean anyDestroyed = false;
|
| try {
|
| while (true) {
|
| boolean destroyed = false;
|
| ConnectionListener cl = null;
|
| synchronized (cls) {
| if (cls.size() == 0) {
| break;
| }
|
| cl = removeForFrequencyCheck();
|
| }
|
| if (cl == null) {
| break;
| }
|
| try {
|
| Set candidateSet =
Collections.singleton(cl.getManagedConnection());
|
| if (mcf instanceof ValidatingManagedConnectionFactory) {
| ValidatingManagedConnectionFactory vcf =
(ValidatingManagedConnectionFactory) mcf;
| candidateSet = vcf.getInvalidConnections(candidateSet);
|
| if (candidateSet != null && candidateSet.size() > 0) {
|
| if (cl.getState() != ConnectionListener.DESTROY) {
| doDestroy(cl);
| destroyed = true;
| anyDestroyed = true;
| }
| }
|
| } else {
| log.warn("warning: background validation was specified
with a non compliant ManagedConnectionFactory interface.");
| }
|
| } finally {
| if (!destroyed) {
| synchronized (cls) {
| returnForFrequencyCheck(cl);
| }
| }
|
| }
|
| }
|
| } finally {
| permits.release();
|
| if (anyDestroyed && shutdown.get() == false &&
poolParams.minSize > 0) {
| PoolFiller.fillPool(this);
| }
|
| }
|
I've had a look at the jira and haven't found this bug as reported. If it has
been already fixed i apologize for making you loose your time, if not, it would
be so kind of you to send me one of those jboss merchandising caps as I have
helped you to find several jca bugs so far... ;) Actually, I think this bug is
so big that I can't understand how nobody has noticed it before...
View the original post :
http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4250999#4250999
Reply to the post :
http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4250999
_______________________________________________
jboss-user mailing list
jboss-user@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-user
