Re: solrj and CLOSE_WAIT's

[Ryan McKinley]

not sure if it is something we can do better or part of HttpClient...
From:
http://www.nabble.com/CLOSE_WAIT-td19959428.html

it seems to suggest you may want to call:
con.closeIdleConnections(0L);

But if you are creating a new MultiThreadedHttpConnectionManager for  
each request, is seems odd you would have to explicitly close the  
connection for each request.

What happens if you try using a SimpleHttpConnectionManager rather  
then a MultiThreadedHttpConnectionManager?  You can explicitly pass in:
  new HttpClient( new SimpleHttpConnectionManager()  )
to:
public CommonsHttpSolrServer(URL baseURL, HttpClient client,  
ResponseParser parser, boolean useMultiPartPost) {

if that fixes things, it is a bit disturbing, but something we should  
look into.

ryan



On Nov 13, 2008, at 7:49 PM, Ian Holsman wrote:

Hi guys.

I'm running a little upload project that uploads documents into a  
solr index. there is also a 2nd thread that runs a deleteby query  
and a optimize every once and a while.

in an effort to reduce the probably of things being held onto I've  
made everything local, but it is still collecting CLOSE_WAITs and  
FIN_WAIT2's on the server side until it eventually runs out of file  
handles in a day or two.

the following are the code snippets being used to call solr.

  protected void doArchiveSolr() throws IOException,  
SolrServerException {
      Calendar rightNow = Calendar.getInstance();
      rightNow.add(Calendar.DATE, 31 * -1);
      DateFormat f = new SimpleDateFormat("yyyy-MM- 
dd'T'HH:mm:ss.SSS'Z'");
      java.util.Date d = rightNow.getTime();

      String s = "publish_date:[1976-03-06T23:59:59.999Z/YEAR TO " +  
f.format(d) + "]";
      logger.info("Archiver:" + s);
      CommonsHttpSolrServer solrServer;
      solrServer = new CommonsHttpSolrServer(solrURL);
      solrServer.deleteByQuery(s);
      solrServer.commit();
  }

and this runs every X minutes.
it also has other local parts like
{
 CommonsHttpSolrServer solrServer;
 solrServer = new CommonsHttpSolrServer(solrURL);
 solrServer.optimize();
}


and
{
                      CommonsHttpSolrServer solrServer;
                      UpdateResponse r;

                      solrServer = new CommonsHttpSolrServer(solrUrl);
                      solrServer.setSoTimeout(120000);  // socket  
read timeout  2minutes
                      solrServer.setConnectionTimeout(100);
                      solrServer.setDefaultMaxConnectionsPerHost(100);
                      solrServer.setMaxTotalConnections(100);
                      solrServer.setFollowRedirects(false);  //  
defaults to false
                      solrServer.setAllowCompression(false);

                      r = solrServer.add(docs);

                      r = solrServer.commit();
                      docs.clear();

}