Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?
That did it, Erik, thanks. On Fri, Jul 1, 2016 at 8:57 PM Erik Hennum wrote: > Hi, Steve: > > Our cookbook examples are maybe a bit misleading in this respect. > > Each example creates its own database client merely so it can stand alone. > > A realistic application would keep a singleton database client until the > application completely finishes using the REST server. To put it the other > way, there's no reason to close a database client if the application is > still using the same REST server as the same user. > > The connection pool is associated with the database client (and, in fact, > is Apache HttpClient in the lower layers), so using a singleton database > client (including sharing across multiple threads) should greatly improve > the socket efficiency. > > > Hoping that helps (and that things are well with you), > > > Erik Hennum > > > -- > *From:* general-boun...@developer.marklogic.com [ > general-boun...@developer.marklogic.com] on behalf of Steve Anderson [ > steve.ander...@gmail.com] > *Sent:* Friday, July 01, 2016 5:02 PM > *To:* MarkLogic Developer Discussion > *Subject:* Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT > under load? > > Thanks for the fast response, Erik, hope you're doing well. > > I am not using a single database client object. The docs say to close() > the client after use. Doing that makes the client unusable for further > transactions, so I've been creating a new one for each request. Should I > have a singleton client and never use close(), or is there some in-between > best practice?? > > I believe I am writing and reading to completion. > > Steve > > On Fri, Jul 1, 2016 at 4:40 PM Erik Hennum > wrote: > >> Hi, Steve: >> >> A few questions: >> >> * Are you creating a single database client object and using it for all >> requests on the REST port? >> >> That's important for connection pooling. The database client is >> thread safe after configuration. >> >> For instance, in a Java servlet environment, the single database >> client would be created during >> application initialization. >> >> * Are you writing and reading to completion? For instance, closing >> streams after reading the data. >> >> >> Hoping that helps, >> >> >> Erik Hennum >> >> >> -- >> *From:* general-boun...@developer.marklogic.com [ >> general-boun...@developer.marklogic.com] on behalf of Steve Anderson [ >> steve.ander...@gmail.com] >> *Sent:* Friday, July 01, 2016 4:09 PM >> *To:* general@developer.marklogic.com >> *Subject:* [MarkLogic Dev General] Sockets being left in CLOSE_WAIT >> under load? >> >> I'm using version 3.0.5 of the Java API client on a Mac with Oracle's >> Java 1.8.0_72 and MarkLogic 8.0-5.1. >> >> I've noticed that my app under load (multiple requests to different end >> points) is leaving of sockets between the server and the database in the >> CLOSE_WAIT state. Under load (thousands of document writes in multiple >> threads), it climbs and climbs and climbs, with no apparent end in sight. >> >> If I force a cleanup via the debugger (System.gc(); >> System.runFinalization ();) the majority of the sockets close. >> >> When I run AllCookbookExamples, and, in another terminal, running "lsof >> -i -n -P|grep _WAIT" I see the same thing; the number of sockets left open >> grows until the session ends. It's no different with the current develop >> branch on github. >> >> I suspect it's something I'm missing related to connection pooling, but I >> don't see any documentation on connection pooling configuration. >> >> I know I can modify ulimit on my system, but I'm afraid it's an >> underlying issue in my code that will keep the sockets open forever and >> eventually cause network issues. >> >> Has anyone else seen anything like this? >> >> Steve >> >> ___ >> General mailing list >> General@developer.marklogic.com >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general >> > ___ > General mailing list > General@developer.marklogic.com > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?
Hi, Steve: Our cookbook examples are maybe a bit misleading in this respect. Each example creates its own database client merely so it can stand alone. A realistic application would keep a singleton database client until the application completely finishes using the REST server. To put it the other way, there's no reason to close a database client if the application is still using the same REST server as the same user. The connection pool is associated with the database client (and, in fact, is Apache HttpClient in the lower layers), so using a singleton database client (including sharing across multiple threads) should greatly improve the socket efficiency. Hoping that helps (and that things are well with you), Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Steve Anderson [steve.ander...@gmail.com] Sent: Friday, July 01, 2016 5:02 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load? Thanks for the fast response, Erik, hope you're doing well. I am not using a single database client object. The docs say to close() the client after use. Doing that makes the client unusable for further transactions, so I've been creating a new one for each request. Should I have a singleton client and never use close(), or is there some in-between best practice?? I believe I am writing and reading to completion. Steve On Fri, Jul 1, 2016 at 4:40 PM Erik Hennum mailto:erik.hen...@marklogic.com>> wrote: Hi, Steve: A few questions: * Are you creating a single database client object and using it for all requests on the REST port? That's important for connection pooling. The database client is thread safe after configuration. For instance, in a Java servlet environment, the single database client would be created during application initialization. * Are you writing and reading to completion? For instance, closing streams after reading the data. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Steve Anderson [steve.ander...@gmail.com<mailto:steve.ander...@gmail.com>] Sent: Friday, July 01, 2016 4:09 PM To: general@developer.marklogic.com<mailto:general@developer.marklogic.com> Subject: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load? I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java 1.8.0_72 and MarkLogic 8.0-5.1. I've noticed that my app under load (multiple requests to different end points) is leaving of sockets between the server and the database in the CLOSE_WAIT state. Under load (thousands of document writes in multiple threads), it climbs and climbs and climbs, with no apparent end in sight. If I force a cleanup via the debugger (System.gc(); System.runFinalization ();) the majority of the sockets close. When I run AllCookbookExamples, and, in another terminal, running "lsof -i -n -P|grep _WAIT" I see the same thing; the number of sockets left open grows until the session ends. It's no different with the current develop branch on github. I suspect it's something I'm missing related to connection pooling, but I don't see any documentation on connection pooling configuration. I know I can modify ulimit on my system, but I'm afraid it's an underlying issue in my code that will keep the sockets open forever and eventually cause network issues. Has anyone else seen anything like this? Steve ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?
Thanks for the fast response, Erik, hope you're doing well. I am not using a single database client object. The docs say to close() the client after use. Doing that makes the client unusable for further transactions, so I've been creating a new one for each request. Should I have a singleton client and never use close(), or is there some in-between best practice?? I believe I am writing and reading to completion. Steve On Fri, Jul 1, 2016 at 4:40 PM Erik Hennum wrote: > Hi, Steve: > > A few questions: > > * Are you creating a single database client object and using it for all > requests on the REST port? > > That's important for connection pooling. The database client is > thread safe after configuration. > > For instance, in a Java servlet environment, the single database > client would be created during > application initialization. > > * Are you writing and reading to completion? For instance, closing > streams after reading the data. > > > Hoping that helps, > > > Erik Hennum > > > -- > *From:* general-boun...@developer.marklogic.com [ > general-boun...@developer.marklogic.com] on behalf of Steve Anderson [ > steve.ander...@gmail.com] > *Sent:* Friday, July 01, 2016 4:09 PM > *To:* general@developer.marklogic.com > *Subject:* [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under > load? > > I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java > 1.8.0_72 and MarkLogic 8.0-5.1. > > I've noticed that my app under load (multiple requests to different end > points) is leaving of sockets between the server and the database in the > CLOSE_WAIT state. Under load (thousands of document writes in multiple > threads), it climbs and climbs and climbs, with no apparent end in sight. > > If I force a cleanup via the debugger (System.gc(); System.runFinalization > ();) the majority of the sockets close. > > When I run AllCookbookExamples, and, in another terminal, running "lsof -i > -n -P|grep _WAIT" I see the same thing; the number of sockets left open > grows until the session ends. It's no different with the current develop > branch on github. > > I suspect it's something I'm missing related to connection pooling, but I > don't see any documentation on connection pooling configuration. > > I know I can modify ulimit on my system, but I'm afraid it's an underlying > issue in my code that will keep the sockets open forever and eventually > cause network issues. > > Has anyone else seen anything like this? > > Steve > > ___ > General mailing list > General@developer.marklogic.com > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?
Hi, Steve: A few questions: * Are you creating a single database client object and using it for all requests on the REST port? That's important for connection pooling. The database client is thread safe after configuration. For instance, in a Java servlet environment, the single database client would be created during application initialization. * Are you writing and reading to completion? For instance, closing streams after reading the data. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Steve Anderson [steve.ander...@gmail.com] Sent: Friday, July 01, 2016 4:09 PM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load? I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java 1.8.0_72 and MarkLogic 8.0-5.1. I've noticed that my app under load (multiple requests to different end points) is leaving of sockets between the server and the database in the CLOSE_WAIT state. Under load (thousands of document writes in multiple threads), it climbs and climbs and climbs, with no apparent end in sight. If I force a cleanup via the debugger (System.gc(); System.runFinalization ();) the majority of the sockets close. When I run AllCookbookExamples, and, in another terminal, running "lsof -i -n -P|grep _WAIT" I see the same thing; the number of sockets left open grows until the session ends. It's no different with the current develop branch on github. I suspect it's something I'm missing related to connection pooling, but I don't see any documentation on connection pooling configuration. I know I can modify ulimit on my system, but I'm afraid it's an underlying issue in my code that will keep the sockets open forever and eventually cause network issues. Has anyone else seen anything like this? Steve ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general