Hey Konstantinos can you sign up for a JIRA account? Then I can grant you permissions to add/update/edit issues.
------------------------ Chris Mattmann chris.mattm...@gmail.com -----Original Message----- From: Konstantinos Mavrommatis <kmavromma...@celgene.com> Reply-To: <dev@oodt.apache.org> Date: Thursday, July 10, 2014 7:11 AM To: "dev@oodt.apache.org" <dev@oodt.apache.org> Subject: RE: UPDATE-PROBABLY SOLVED: File manager keeps connections to SOLR in CLOSE_WAIT state for hours >Unless I am missing something I am not able to create an issue in JIRA :( >Thanks >K > >> -----Original Message----- >> From: Ramirez, Paul M (398J) [mailto:paul.m.rami...@jpl.nasa.gov] >> Sent: Thursday, July 10, 2014 4:06 PM >> To: <dev@oodt.apache.org> >> Subject: Re: UPDATE-PROBABLY SOLVED: File manager keeps connections to >> SOLR in CLOSE_WAIT state for hours >> >> Konstantinos, >> >> Could you open a Jira issue for this and attach a patch? This would be >> helpful for the project. >> >> https://issues.apache.org/jira/browse/OODT/?selectedTab=com.atlassian.j >> ira.jira-projects-plugin:summary-panel >> >> >> Thanks, >> Paul Ramirez >> >> > On Jul 10, 2014, at 5:23 AM, "Konstantinos Mavrommatis" >> <kmavromma...@celgene.com> wrote: >> > >> > Hi, >> > >> > Following the suggestion at >> > >> > http://blogs.nuxeo.com/development/2013/02/using-httpclient-properly- >> a >> > void-closewait-tcp-connections/ >> > >> > >> > >> > I modified the code in >> > >> > >> src/main/java/org/apache/oodt/cas/filemgr/catalog/solr/SolrClient.java >> > >> > >> > >> > and added the line highlighted in red >> (method.setRequestHeader("Connection","close"). >> > >> > This seems to have resolved the problem at least until now ingestion >> of few thousand files have not produced any stale connection. >> > >> > Please let me know your thoughts on this solution. >> > >> > Best, >> > >> > Konstantinos >> > >> > >> > >> > ====================================================== >> > >> > private String doHttp(HttpMethod method) throws Exception { >> > >> > >> > >> > StringBuilder response = new StringBuilder(); >> > >> > BufferedReader br = null; >> > >> > try { >> > >> > >> > >> > // send request >> > >> > HttpClient httpClient = new HttpClient(); >> > >> > // 10 July 2014. attempting to avoid problems >> > with CLOSE_WAIT connections Based on >> > >> > >> > //http://blogs.nuxeo.com/development/2013/02/using-httpclient- >> properly >> > -avoid-closewait-tcp-connections/ >> > >> > method.setRequestHeader("Connection", >> "close"); >> > >> > int statusCode = httpClient.executeMethod(method); >> > >> > >> > >> > // read response >> > >> > if (statusCode != HttpStatus.SC_OK) { >> > >> > >> > >> > // still consume the response >> > >> > method.getResponseBodyAsString(); >> > >> > throw new CatalogException("HTTP method failed: " + >> > method.getStatusLine()); >> > >> > >> > >> > } else { >> > >> > >> > >> > // read the response body. >> > >> > br = new BufferedReader(new >> > InputStreamReader(method.getResponseBodyAsStream())); >> > >> > String readLine; >> > >> > while(((readLine = br.readLine()) != null)) { >> > >> > response.append(readLine); >> > >> > } >> > >> > >> > >> > ======================================================= >> > >> > >> > >> > Information about the system: >> > >> > OS is MacOS Mavericks, >> > >> > SOLR version : 4.6.1 , SOLR runs on a single server, no replication >> at all. >> > >> > Tomcat verision: 7.0.50 >> > >> > OODT version is 0.6 >> > >> > Java(TM) SE Runtime Environment (build 1.7.0_51-b13) >> > >> > >> > >> > >> > >> > >> > >> >> -----Original Message----- >> > >> >> From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] >> > >> >> Sent: Wednesday, July 09, 2014 4:57 AM >> > >> >> To: dev@oodt.apache.org >> > >> >> Subject: Re: File manager keeps connections to SOLR in CLOSE_WAIT >> >> state >> > >> >> for hours >> > >> > >> >> It's just occoured to me that everything in >> > >> >> org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog (well, >> > >> >> specifically >> > >> >> org.apache.oodt.cas.filemgr.catalog.solr.SolrClient) is done through >> > >> >> HTTPClient so all of the above may not be relevant. >> > >> >> Can you please scope all the same? >> > >> >> Thanks >> > >> >> Lewis >> > >> > >> > >> >> On Tue, Jul 8, 2014 at 10:54 PM, Lewis John Mcgibbney < >> > >> >> lewis.mcgibb...@gmail.com<mailto:lewis.mcgibb...@gmail.com>> wrote: >> > >> > >> >>> Hi Konstantinos, >> > >> >>> OK, I was ablel to scope this one and I have a few questions for >> you. >> > >> >>> 1) Which version of Solr are you using? Is it 3.5, 3.6, 4.0-ALPHA? >> > >> >> If >> > >> >>> so please scope this issue [0], the solution would be to upgrade if >> > >> >>> you are not too long ahead with ingestion as fixes in Solr are >> worth >> > >> >>> having based on recent release cycles. >> > >> >>> 2) How many cores do you have on Solr server? Also what kind of >> >>> setUp >> > >> >>> do you have? Replication at all? >> > >> >>> In recent versions of Solr 4.X SolrJ clients should now call >> > >> >>> shutdown() on their SolrServer object to let it know they don't >> want >> > >> >>> to re-use any existing connections anymore, and when Solr >> internally >> > >> >>> uses SolrJ to talk to other nodes in SolrCloud it should be doing >> > >> >> this >> > >> >>> (as of 4.0-ALPHA)so this is why I ask. >> > >> >>> Lewis >> > >> > >> >>> [0] https://issues.apache.org/jira/browse/SOLR-3280 >> > >> > >> > >> >>> On Tue, Jul 8, 2014 at 7:14 AM, Konstantinos Mavrommatis < >> > >> >>> kmavromma...@celgene.com<mailto:kmavromma...@celgene.com>> wrote: >> > >> > >> >>>> Hi, >> > >> >>>> I have setup OODT filemanager on port 9000, using SOLR as the >> > >> >>>> indexing service on port 8081. They are both setup on the same >> > >> >>>> computer, while crawler runs on a number of different compute >> nodes >> > >> >>>> spread across the local network and the cloud. >> > >> > >> >>>> When the crawler runs and ingests files I notice that there are >> > >> >>>> several connections that open to solr and remain in CLOSE_WAIT >> >>>> state >> > >> >> for hours. >> > >> >>>> any idea why this happens? Moving forward I am planning to use >> > >> >>>> several hundreds of crawler instances, each running on different >> > >> >>>> computer, that will create thousands of such connections and will >> > >> >>>> probably create problems to the system. >> > >> >>>> Thanks in advance for any help >> > >> >>>> Kostas >> > >> > >> >>>> $lsof -i :8081 >> > >> >>>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF >> > >> >> NODE >> > >> >>>> NAME >> > >> >>>> java 92065 kmavrommatis 75u IPv6 0x392c3fa3b63b29cf 0t0 >> > >> >> TCP >> > >> >>>> localhost:49205->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 76u IPv6 0x392c3fa3b6dcbbcf 0t0 >> > >> >> TCP >> > >> >>>> localhost:49206->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 77u IPv6 0x392c3fa39fd12e0f 0t0 >> > >> >> TCP >> > >> >>>> localhost:49207->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 78u IPv6 0x392c3fa39fdcdbcf 0t0 >> > >> >> TCP >> > >> >>>> localhost:49208->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 79u IPv6 0x392c3fa3b62cde0f 0t0 >> > >> >> TCP >> > >> >>>> localhost:49209->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 80u IPv6 0x392c3fa39fa2714f 0t0 >> > >> >> TCP >> > >> >>>> localhost:49210->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 81u IPv6 0x392c3fa3b6c32acf 0t0 >> > >> >> TCP >> > >> >>>> localhost:49211->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> >>>> java 92065 kmavrommatis 82u IPv6 0x392c3fa3b6aa714f 0t0 >> > >> >> TCP >> > >> >>>> localhost:49212->localhost:sunproxyadmin (CLOSE_WAIT) >> > >> > >> > >> >>>> process 92065 is: >> > >> >>>> /usr/bin/java -Djava.ext.dirs=../lib >> > >> >>>> -Djava.util.logging.config.file=../etc/logging.properties >> > >> >>>> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties >> > >> >>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager --portNum >> 9000 >> > >> > >> >>>> ********************************************************* >> > >> >>>> THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS CONFIDENTIAL >> AND >> > >> >>>> MAY CONTAIN LEGALLY PRIVILEGED INFORMATION INTENDED ONLY FOR THE >> >>>> USE >> > >> >>>> OF THE INDIVIDUAL OR INDIVIDUALS NAMED ABOVE. >> > >> >>>> If the reader is not the intended recipient, or the employee or >> > >> >> agent >> > >> >>>> responsible to deliver it to the intended recipient, you are >> hereby >> > >> >>>> notified that any dissemination, distribution or copying of this >> > >> >>>> communication is strictly prohibited. If you have received this >> > >> >>>> communication in error, please reply to the sender to notify us of >> > >> >>>> the error and delete the original message. Thank You. >> > >> > >> > >> > >> > >> >>> -- >> > >> >>> *Lewis* >> > >> > >> > >> > >> > >> >> -- >> > >> >> *Lewis* >> > >> > ********************************************************* >> > THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS CONFIDENTIAL AND >> > MAY CONTAIN LEGALLY PRIVILEGED INFORMATION INTENDED ONLY FOR THE USE >> > OF THE INDIVIDUAL OR INDIVIDUALS NAMED ABOVE. >> > If the reader is not the intended recipient, or the employee or agent >> > responsible to deliver it to the intended recipient, you are hereby >> > notified that any dissemination, distribution or copying of this >> > communication is strictly prohibited. If you have received this >> > communication in error, please reply to the sender to notify us of >> the >> > error and delete the original message. Thank You. > >********************************************************* >THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS >CONFIDENTIAL AND MAY CONTAIN LEGALLY PRIVILEGED >INFORMATION INTENDED ONLY FOR THE USE OF THE INDIVIDUAL >OR INDIVIDUALS NAMED ABOVE. >If the reader is not the intended recipient, or the >employee or agent responsible to deliver it to the >intended recipient, you are hereby notified that any >dissemination, distribution or copying of this >communication is strictly prohibited. If you have >received this communication in error, please reply to the >sender to notify us of the error and delete the original >message. Thank You.
