On Wed, 2011-01-19 at 02:32 -0800, olze wrote:
> I wrote a multithreaded Tool which creates some DefaultHttpClient instances
> and HttpGets which then download a site (like a crawler). My Problem is
> that, after a couple of minutes (~20) the application goes slower and
> slower. In the beginning 10 Threads are running, and after about 20 minutes
> only one single thread is running. So i took a look at the debugging window
> in netbeans and saw that all other threads are hanging in
> ThreadSafeClientConnManager.getConnection:216
>
> i made a screenshot with the thread trace:
> http://www.imagebanana.com/view/zljfdwh4/screenshot.PNG
>
> I already changed the default max. connections per route to 20.
>
> the source i use (initialisation):
> schemeRegistry = new SchemeRegistry();
> schemeRegistry.register(
> new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
> schemeRegistry.register(
> new Scheme("https", 443, SSLSocketFactory.getSocketFactory()));
>
> cm = new ThreadSafeClientConnManager(schemeRegistry);
> cm.closeIdleConnections(4, TimeUnit.SECONDS);
> cm.closeExpiredConnections();
> cm.setMaxTotal(200);
> cm.setDefaultMaxPerRoute(100);
>
> the usage (threads):
> DefaultHttpClient client = new DefaultHttpClient(Main.cm, Main.params);
> client.setRedirectHandler(new CustomRedirectHandler());
> client.getParams().setParameter(ClientPNames.COOKIE_POLICY,
>
> org.apache.http.client.params.CookiePolicy.BROWSER_COMPATIBILITY);
>
> HttpResponse response = null;
>
> try {
> response = client.execute(get);
> } catch (Exception ex) {
No need to do that. HttpClient automatically cleans up in case of an I/O
error or runtime error in the #execute method.
> get.abort();
> }
>
> entity = response.getEntity();
no need to do all that. If there is no entity, there are no resources
that need to be deallocated
---
> if (entity == null) {
> get.abort();
> pageURL = null;
> client = null;
> return;
> }
>
---
> fos = new FileOutputStream(new File(filename));
> try {
> entity.writeTo(fos);
> } catch (Exception ex) {
> ex.printStackTrace();
> if (entity != null) {
> EntityUtils.consume(entity);
> }
> }
>
> this is not the complete code, i just copied the important parts. i always
> use EntityUtils.consume() and get.abort if i see something unexpected
> happened, so i think all resources should be free'd?
>
Most likely your application still leaks connections somewhere.
> Is there any way i can check if the resources are 100% free'd or if there is
> some leak?
>
Run your application with context logging for connection management /
request execution enabled and see if connections get correctly returned
to the pool.
http://hc.apache.org/httpcomponents-client-dev/logging.html
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
