[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Haroon (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741318#action_12741318
 ] 

Haroon commented on AXIS2-2931:
---

We are facing a similar issue with Axis2-1.2 version when we use Axis2 as our 
Webservices client to make WS calls.

Exception stack trace is : org.apache.axis2.AxisFault: Too many open files
Caused by: org.apache.axis2.AxisFault: Too many open files
Caused by: java.net.SocketException: Too many open files

I saw this post here on AXIS2-2931 about the patch for this fix.  And it is 
mentioned that it can be seamlessly applied in axis2-1.3 and 1.4



However, since we are on Axis2-1.2 version, could you please let us know the 
approach for fixing this issue, with minimum changes to the list of axis2 jars?

I'm doubtful, if I take the fix from 
 and 
replace axis2-kernel-1.2.jar with axis2-kernel-1.3i3.jar, what other dependent 
jars I might need to upgrade?

Any sort of help/guidelines on applying the proper fix to axis2-1.2 version is 
appreciated.

> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Priority: Critical
> Attachments: 
> 0001-AXIS2-2931-AXIS2-3478-Release-the-HTTP-connection.patch
>
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boolean success = true;
>   while(success) {
>   success = stub.performService(records);
>   stub.cleanup();
>   //  System.gc();
>   }
> Here is the complete stack trace:
>   Exception in thread "main" org.apache.axis2.AxisFault: Address already 
> in use: connect
>   at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
>   at 
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
>   at 
> org.apache.axis2.transp

[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Alexis Midon (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677625#action_12677625
 ] 

Alexis Midon commented on AXIS2-2931:
-

a related thread on axis2 ml:
http://markmail.org/thread/voabzcl6u2pck74n

> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Priority: Critical
> Attachments: 
> 0001-AXIS2-2931-AXIS2-3478-Release-the-HTTP-connection.patch
>
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boolean success = true;
>   while(success) {
>   success = stub.performService(records);
>   stub.cleanup();
>   //  System.gc();
>   }
> Here is the complete stack trace:
>   Exception in thread "main" org.apache.axis2.AxisFault: Address already 
> in use: connect
>   at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
>   at 
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
>   at 
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:310)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:202)
>   at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:446)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:330)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
>   at 
> com.ws.service.MyServiceStub.performService(MyServiceStub.java:300)
>   at com.ws.client.MyClient.main(MyClient.java:88)
>   Caused by: java.net.BindException: Address already in use: connect
>   at java.net.PlainSocketImpl.socketConnect(Native Method)
>   at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
>   

[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Andreas Veithen (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677308#action_12677308
 ] 

Andreas Veithen commented on AXIS2-2931:


Reading the entire response into memory is a performance issue for two reasons:
* Axis2 would no longer be able to start processing the response before the 
entire message has been received.
* The raw content of the entire message would have to be kept in memory. The 
argument that "an xml parser will have to do it at some point anyway" is not 
valid. Even if the full Axiom tree of the message is built (which is not 
necessarily the case), there is no justification for having two copies of the 
message in memory (one in the form of a byte array, the other in the form of an 
Axiom tree).

> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Priority: Critical
> Attachments: 
> 0001-AXIS2-2931-AXIS2-3478-Release-the-HTTP-connection.patch
>
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boolean success = true;
>   while(success) {
>   success = stub.performService(records);
>   stub.cleanup();
>   //  System.gc();
>   }
> Here is the complete stack trace:
>   Exception in thread "main" org.apache.axis2.AxisFault: Address already 
> in use: connect
>   at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
>   at 
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
>   at 
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:310)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:202)
>   at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:446)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperat

[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Alexis Midon (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676360#action_12676360
 ] 

Alexis Midon commented on AXIS2-2931:
-

I have the exact same issue with the default settings (axis 1.3): unclosed 
connection and too many open files.

I tried the options REUSE_HTTP_CLIENT and AUTO_RELEASE_CONNECTION and drew the 
exact same conclusions as Guillaume [1]

. REUSE_HTTP_CLIENT starves the pool and invocations hang out
. AUTO_RELEASE_CONNECTION release the connection, but before the SOAP handler 
had a chance to read the http input stream :( 


[1] 
https://issues.apache.org/jira/browse/AXIS2-935?focusedCommentId=12513543#action_12513543

> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Priority: Critical
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boolean success = true;
>   while(success) {
>   success = stub.performService(records);
>   stub.cleanup();
>   //  System.gc();
>   }
> Here is the complete stack trace:
>   Exception in thread "main" org.apache.axis2.AxisFault: Address already 
> in use: connect
>   at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
>   at 
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
>   at 
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:310)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:202)
>   at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:446)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:330)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
>   at 
> com.ws.service.MyServ

[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Tobias Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542809
 ] 

Tobias Jordan commented on AXIS2-2931:
--

I am sorry, but the issue still exists in 1.3. I tried to set the 
AUTO_RELEASE_CONNECTION option in a blocking operations client (together with 
REUSE_HTTP_CLIENT). This results in 'Attempted read on closed stream' 
exceptions.

> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Priority: Critical
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boolean success = true;
>   while(success) {
>   success = stub.performService(records);
>   stub.cleanup();
>   //  System.gc();
>   }
> Here is the complete stack trace:
>   Exception in thread "main" org.apache.axis2.AxisFault: Address already 
> in use: connect
>   at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
>   at 
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
>   at 
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:310)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:202)
>   at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:446)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:330)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
>   at 
> com.ws.service.MyServiceStub.performService(MyServiceStub.java:300)
>   at com.ws.client.MyClient.main(MyClient.java:88)
>   Caused by: java.net.BindException: Address already in use: connect
>   at java.net.PlainSocketImpl.socketConnect(Native Method)
>   at java.net.PlainSocketImpl.doConnect(

[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Deepal Jayasinghe (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541901
 ] 

Deepal Jayasinghe commented on AXIS2-2931:
--

Please test with Axis2 1.3 , I think we have fixed the issue in 1.3 release.

Thanks
Deepal

> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Priority: Critical
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boolean success = true;
>   while(success) {
>   success = stub.performService(records);
>   stub.cleanup();
>   //  System.gc();
>   }
> Here is the complete stack trace:
>   Exception in thread "main" org.apache.axis2.AxisFault: Address already 
> in use: connect
>   at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
>   at 
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
>   at 
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:310)
>   at 
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:202)
>   at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:446)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:330)
>   at 
> org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
>   at 
> com.ws.service.MyServiceStub.performService(MyServiceStub.java:300)
>   at com.ws.client.MyClient.main(MyClient.java:88)
>   Caused by: java.net.BindException: Address already in use: connect
>   at java.net.PlainSocketImpl.socketConnect(Native Method)
>   at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
>   at 
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
>

[jira] Commented: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

[Gerry Holly (JIRA)]

[ 
https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511321
 ] 

Gerry Holly commented on AXIS2-2931:


Hello,

It seems the AUTO_RELEASE_CONNECTION solution only covers non-blocking 
operations. What about blocking operations?

In a blocking scenario, when "handleResponse" is invoked in HTTPSender, it 
stores a reference to the InputStream in the message context so it can be later 
processed by the application. Your fix renders this InputStream unreadable 
before the OutInAxisOperationClient gets a chance to process the SOAP message 
(through TransportUtils).

This is the exception I get after implementing your HTTPConstants.java and 
HTTPSender.java (not using the nightly build):

Caused by: org.apache.axis2.AxisFault: Attempted read on closed stream.
at org.apache.axis2.builder.SOAPBuilder.processDocument(SOAPBuilder.java:51)
at 
org.apache.axis2.transport.TransportUtils.createSOAPMessage(TransportUtils.java:130)
at 
org.apache.axis2.transport.TransportUtils.createSOAPMessage(TransportUtils.java:77)
at 
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:356)
at 
org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
... 7 more
Caused by: java.io.IOException: Attempted read on closed stream.
at 
org.apache.commons.httpclient.AutoCloseInputStream.isReadAllowed(AutoCloseInputStream.java:165)
at 
org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:106)
at java.io.FilterInputStream.read(Unknown Source)
at java.io.PushbackInputStream.read(Unknown Source)
at org.apache.axis2.builder.BuilderUtil.getReader(BuilderUtil.java:191)
at org.apache.axis2.builder.SOAPBuilder.processDocument(SOAPBuilder.java:41)
... 11 more

Gerry


> CLONE -Web Service request loop causes many TIME_WAIT Connections & 
> "BindException: Address already in use: connect"
> 
>
> Key: AXIS2-2931
> URL: https://issues.apache.org/jira/browse/AXIS2-2931
> Project: Axis 2.0 (Axis2)
>  Issue Type: Bug
>Affects Versions: 1.1.1, nightly
> Environment: Windows XP, Tomcat, Axis2 nightly build (also 
> encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 
> 126 GB free disk space, 1GB RAM.
>Reporter: Gerry Holly
>Assignee: Davanum Srinivas
>Priority: Critical
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed 
> even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated 
> caused by the following exception (see below for full stack trace):
>   java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but 
> not always) remain in the TIME_WAIT state for several minutes.  There are 
> 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does 
> not appear to occur, as the system eventually releases the connections after 
> a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a 
> message to the web service (the commented out line of code below), the 
> AxisFault does not occur.  I have executed the loop over 60,000 times with 
> the garbage collection included without generating an AxisFault.  There were 
> only 400-700 connections in the TIME_WAIT state at a given time, but 
> performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
>   HttpConnectionManagerParams connectionManagerParams = new 
> HttpConnectionManagerParams();
>   connectionManagerParams.setTcpNoDelay(true);
>   connectionManagerParams.setStaleCheckingEnabled(true);
>   
>   MultiThreadedHttpConnectionManager connectionManager = new 
> MultiThreadedHttpConnectionManager();
>   connectionManager.setParams(connectionManagerParams);
>   HttpClient httpClient = new HttpClient(connectionManager);
>   
>   ConfigurationContext configurationContext = 
> ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, 
> null);
>   configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, 
> Boolean.TRUE);
>   configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, 
> httpClient);
>   
>   stub = new MyServiceStub(configurationContext, target);
>   
>   boo