RE: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-10-19 Thread Markus Jelsma
By the way, we also see a generous amount of warnings in Zookeeper's logs. Are 
these related? An indication of what?

Thanks,
Markus

2017-10-19 08:57:35,583 [myid:2] - WARN  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of 
stream exception
EndOfStreamException: Unable to read additional data from client sessionid 
0x15e1925fb7e3748, likely client has closed socket
at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
at java.lang.Thread.run(Thread.java:748)
2017-10-19 08:57:35,583 [myid:2] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket 
connection for client /xxx.xxx.xxx.xxx:41312 which had sessionid 
0x15e1925fb7e3748


 
 
-Original message-
> From:Markus Jelsma 
> Sent: Thursday 19th October 2017 13:45
> To: solr-user@lucene.apache.org
> Subject: RE: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> 
> Hello,
> 
> We are having this problem again, now it affects the front-end too, the logs 
> are littered with Zookeeper connection log lines at WARN level.
> 
> Is it expected that i have to deal with this problem myself? Isn't SolrJ or 
> HTTPClient even going to guarantee me that they will handle underlying 
> connection problems?
> 
> If i have to deal with it myself, is it just a case of catching 
> IllegalStateException and closing and reconnecting SolrClient?
> 
> Thanks,
> Markus
> 
> -Original message-
> > From:Shawn Heisey 
> > Sent: Tuesday 18th July 2017 16:18
> > To: solr-user@lucene.apache.org
> > Subject: Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> > 
> > On 7/18/2017 5:10 AM, Markus Jelsma wrote:
> > > The problem was never resolved but Shawn asked for the stack trace, here 
> > > it is:
> > 
> > > Caused by: java.lang.IllegalStateException: Connection pool shut down 
> > > at org.apache.http.util.Asserts.check(Asserts.java:34) 
> > 
> > As I suspected, it is the connection pool inside HttpClient that is shut
> > down (closed).
> > 
> > Earlier today before I came into the office, I asked the HttpClient user
> > list whether this could ever happen for a reason other than an explicit
> > close/shutdown.  They looked at the code and found that the exception
> > only is thrown if the "isShutDown" boolean flag is true, and the only
> > place that ever gets set to true is when an explicit shutdown is called
> > on the connection pool.
> > 
> > When a solr client is built without an external HttpClient, calling
> > close() on the solr client will shut down the internal HttpClient.  If
> > an external HttpClient is used, the user code would need to shut it down
> > for this to happen.  Recent versions of SolrJ are using
> > CloseableHttpClient, which will shut down the connection pool if close()
> > is called.
> > 
> > It's looking like this error has happened because the HttpClient object
> > inside the solr client has been shut down explicitly, which might have
> > happened because one of the outer layers had close() called.
> > 
> > Thanks,
> > Shawn
> > 
> > 
> 


RE: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-10-19 Thread Markus Jelsma
Hello,

We are having this problem again, now it affects the front-end too, the logs 
are littered with Zookeeper connection log lines at WARN level.

Is it expected that i have to deal with this problem myself? Isn't SolrJ or 
HTTPClient even going to guarantee me that they will handle underlying 
connection problems?

If i have to deal with it myself, is it just a case of catching 
IllegalStateException and closing and reconnecting SolrClient?

Thanks,
Markus

-Original message-
> From:Shawn Heisey 
> Sent: Tuesday 18th July 2017 16:18
> To: solr-user@lucene.apache.org
> Subject: Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> 
> On 7/18/2017 5:10 AM, Markus Jelsma wrote:
> > The problem was never resolved but Shawn asked for the stack trace, here it 
> > is:
> 
> > Caused by: java.lang.IllegalStateException: Connection pool shut down 
> > at org.apache.http.util.Asserts.check(Asserts.java:34) 
> 
> As I suspected, it is the connection pool inside HttpClient that is shut
> down (closed).
> 
> Earlier today before I came into the office, I asked the HttpClient user
> list whether this could ever happen for a reason other than an explicit
> close/shutdown.  They looked at the code and found that the exception
> only is thrown if the "isShutDown" boolean flag is true, and the only
> place that ever gets set to true is when an explicit shutdown is called
> on the connection pool.
> 
> When a solr client is built without an external HttpClient, calling
> close() on the solr client will shut down the internal HttpClient.  If
> an external HttpClient is used, the user code would need to shut it down
> for this to happen.  Recent versions of SolrJ are using
> CloseableHttpClient, which will shut down the connection pool if close()
> is called.
> 
> It's looking like this error has happened because the HttpClient object
> inside the solr client has been shut down explicitly, which might have
> happened because one of the outer layers had close() called.
> 
> Thanks,
> Shawn
> 
> 


Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-07-18 Thread Shawn Heisey
On 7/18/2017 5:10 AM, Markus Jelsma wrote:
> The problem was never resolved but Shawn asked for the stack trace, here it 
> is:

> Caused by: java.lang.IllegalStateException: Connection pool shut down 
> at org.apache.http.util.Asserts.check(Asserts.java:34) 

As I suspected, it is the connection pool inside HttpClient that is shut
down (closed).

Earlier today before I came into the office, I asked the HttpClient user
list whether this could ever happen for a reason other than an explicit
close/shutdown.  They looked at the code and found that the exception
only is thrown if the "isShutDown" boolean flag is true, and the only
place that ever gets set to true is when an explicit shutdown is called
on the connection pool.

When a solr client is built without an external HttpClient, calling
close() on the solr client will shut down the internal HttpClient.  If
an external HttpClient is used, the user code would need to shut it down
for this to happen.  Recent versions of SolrJ are using
CloseableHttpClient, which will shut down the connection pool if close()
is called.

It's looking like this error has happened because the HttpClient object
inside the solr client has been shut down explicitly, which might have
happened because one of the outer layers had close() called.

Thanks,
Shawn



RE: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-07-18 Thread Markus Jelsma
Hello Susheel,

Yes, the closing happens only at the end of the checking cycle. I asked my 
colleague about the firewall and he is positive everything is allowed between 
those nodes. I also cannot completely drop the firewall between those nodes to 
be sure, because the problem is very hard to reproduce; it pops up once in a 
while, sometimes not for weeks, today already a couple of times.

It is locally unreproducible but we're going to try to reproduce it in our 
development environment. So i have to get back to the problem in a few weeks 
from now.

Number of requests is sometimes, briefly, very high. Usually very low. These 
specific checks are executed in order, not concurrently.

Thanks,
Markus
 
-Original message-
> From:Susheel Kumar 
> Sent: Tuesday 18th July 2017 15:17
> To: solr-user@lucene.apache.org
> Subject: Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> 
> Then most likely its due to closing of connection as mentioned above though
> you said it's not happening in that part of your code.  To rule out
> firewall possibility, you can test in some other/local env.  Also how many
> requests/client/connections happening concurrently.
> 
> Thanks,
> Susheel
> 
> On Tue, Jul 18, 2017 at 8:43 AM, Markus Jelsma 
> wrote:
> 
> > Hello Susheel,
> >
> > No, nothing at all. I've check all six nodes, they are clean.
> >
> > Thanks,
> > Markus
> >
> >
> >
> > -Original message-
> > > From:Susheel Kumar 
> > > Sent: Tuesday 18th July 2017 14:30
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> > >
> > > Do you see any errors etc. in solr.log during this time?
> > >
> > > On Tue, Jul 18, 2017 at 7:10 AM, Markus Jelsma <
> > markus.jel...@openindex.io>
> > > wrote:
> > >
> > > > The problem was never resolved but Shawn asked for the stack trace,
> > here
> > > > it is:
> > > >
> > > > org.apache.solr.client.solrj.SolrServerException: java.lang.
> > IllegalStateException:
> > > > Connection pool shut down
> > > > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> > > > doRequest(LBHttpSolrClient.java:485)
> > > > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(
> > > > LBHttpSolrClient.java:388)
> > > > at org.apache.solr.client.solrj.impl.CloudSolrClient.
> > > > sendRequest(CloudSolrClient.java:1383)
> > > > at org.apache.solr.client.solrj.impl.CloudSolrClient.
> > > > requestWithRetryOnStaleState(CloudSolrClient.java:1134)
> > > > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(
> > > > CloudSolrClient.java:1073)
> > > > at org.apache.solr.client.solrj.SolrRequest.process(
> > SolrRequest.java:160)
> > > > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
> > > > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.
> > java:1173)
> > > > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.
> > java:1090)
> > > > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.
> > java:1110)
> > > > ..internal method calling getById()..
> > > > at java.lang.Thread.run(Thread.java:748)
> > > > Caused by: java.lang.IllegalStateException: Connection pool shut down
> > > > at org.apache.http.util.Asserts.check(Asserts.java:34)
> > > > at org.apache.http.pool.AbstractConnPool.lease(
> > AbstractConnPool.java:184)
> > > > at org.apache.http.pool.AbstractConnPool.lease(
> > AbstractConnPool.java:217)
> > > > at org.apache.http.impl.conn.PoolingClientConnectionManager
> > > > .requestConnection(PoolingClientConnectionManager.java:184)
> > > > at org.apache.http.impl.client.DefaultRequestDirector.execute(
> > > > DefaultRequestDirector.java:415)
> > > > at org.apache.http.impl.client.AbstractHttpClient.doExecute(
> > > > AbstractHttpClient.java:882)
> > > > at org.apache.http.impl.client.CloseableHttpClient.execute(
> > > > CloseableHttpClient.java:82)
> > > > at org.apache.http.impl.client.CloseableHttpClient.execute(
> > > > CloseableHttpClient.java:55)
> > > > at org.apache.solr.client.solrj.impl.HttpSolrClient.
> > > > executeMethod(HttpSolrClient.java:515)
> > > > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> > > > HttpSolrClient.java:279)
> > > > at org.apache.solr.c

Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-07-18 Thread Susheel Kumar
Then most likely its due to closing of connection as mentioned above though
you said it's not happening in that part of your code.  To rule out
firewall possibility, you can test in some other/local env.  Also how many
requests/client/connections happening concurrently.

Thanks,
Susheel

On Tue, Jul 18, 2017 at 8:43 AM, Markus Jelsma 
wrote:

> Hello Susheel,
>
> No, nothing at all. I've check all six nodes, they are clean.
>
> Thanks,
> Markus
>
>
>
> -Original message-
> > From:Susheel Kumar 
> > Sent: Tuesday 18th July 2017 14:30
> > To: solr-user@lucene.apache.org
> > Subject: Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> >
> > Do you see any errors etc. in solr.log during this time?
> >
> > On Tue, Jul 18, 2017 at 7:10 AM, Markus Jelsma <
> markus.jel...@openindex.io>
> > wrote:
> >
> > > The problem was never resolved but Shawn asked for the stack trace,
> here
> > > it is:
> > >
> > > org.apache.solr.client.solrj.SolrServerException: java.lang.
> IllegalStateException:
> > > Connection pool shut down
> > > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> > > doRequest(LBHttpSolrClient.java:485)
> > > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(
> > > LBHttpSolrClient.java:388)
> > > at org.apache.solr.client.solrj.impl.CloudSolrClient.
> > > sendRequest(CloudSolrClient.java:1383)
> > > at org.apache.solr.client.solrj.impl.CloudSolrClient.
> > > requestWithRetryOnStaleState(CloudSolrClient.java:1134)
> > > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(
> > > CloudSolrClient.java:1073)
> > > at org.apache.solr.client.solrj.SolrRequest.process(
> SolrRequest.java:160)
> > > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
> > > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.
> java:1173)
> > > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.
> java:1090)
> > > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.
> java:1110)
> > > ..internal method calling getById()..
> > > at java.lang.Thread.run(Thread.java:748)
> > > Caused by: java.lang.IllegalStateException: Connection pool shut down
> > > at org.apache.http.util.Asserts.check(Asserts.java:34)
> > > at org.apache.http.pool.AbstractConnPool.lease(
> AbstractConnPool.java:184)
> > > at org.apache.http.pool.AbstractConnPool.lease(
> AbstractConnPool.java:217)
> > > at org.apache.http.impl.conn.PoolingClientConnectionManager
> > > .requestConnection(PoolingClientConnectionManager.java:184)
> > > at org.apache.http.impl.client.DefaultRequestDirector.execute(
> > > DefaultRequestDirector.java:415)
> > > at org.apache.http.impl.client.AbstractHttpClient.doExecute(
> > > AbstractHttpClient.java:882)
> > > at org.apache.http.impl.client.CloseableHttpClient.execute(
> > > CloseableHttpClient.java:82)
> > > at org.apache.http.impl.client.CloseableHttpClient.execute(
> > > CloseableHttpClient.java:55)
> > > at org.apache.solr.client.solrj.impl.HttpSolrClient.
> > > executeMethod(HttpSolrClient.java:515)
> > > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> > > HttpSolrClient.java:279)
> > > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> > > HttpSolrClient.java:268)
> > > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> > > doRequest(LBHttpSolrClient.java:447)
> > > ... 24 more
> > >
> > > So, to summarize, we have a program checking presence of documents in
> Solr
> > > using getById() and we don't want this exception to bubble up, we want
> > > SolrJ to restore the connection pool just as CloudSolrClient would
> move on
> > > to another node if one went down in the mean time.
> > >
> > > Is this possible? How?
> > >
> > > Many thanks,
> > > Markus
> > >
> > > -Original message-
> > > > From:Markus Jelsma 
> > > > Sent: Thursday 29th June 2017 16:38
> > > > To: solr-user@lucene.apache.org
> > > > Subject: RE: SolrJ 6.6.0 Connection pool shutdown
> > > >
> > > > Thanks. I probably should have mentioned there is no firewall
> limiting
> > > connections between those hosts. Actually, the processes run on the
> same
> > > hosts as the Solr cluster is running on.
> > > >
> > > > Thanks,
> > > > M

RE: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-07-18 Thread Markus Jelsma
Hello Susheel,

No, nothing at all. I've check all six nodes, they are clean.

Thanks,
Markus

 
 
-Original message-
> From:Susheel Kumar 
> Sent: Tuesday 18th July 2017 14:30
> To: solr-user@lucene.apache.org
> Subject: Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace
> 
> Do you see any errors etc. in solr.log during this time?
> 
> On Tue, Jul 18, 2017 at 7:10 AM, Markus Jelsma 
> wrote:
> 
> > The problem was never resolved but Shawn asked for the stack trace, here
> > it is:
> >
> > org.apache.solr.client.solrj.SolrServerException: 
> > java.lang.IllegalStateException:
> > Connection pool shut down
> > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> > doRequest(LBHttpSolrClient.java:485)
> > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(
> > LBHttpSolrClient.java:388)
> > at org.apache.solr.client.solrj.impl.CloudSolrClient.
> > sendRequest(CloudSolrClient.java:1383)
> > at org.apache.solr.client.solrj.impl.CloudSolrClient.
> > requestWithRetryOnStaleState(CloudSolrClient.java:1134)
> > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(
> > CloudSolrClient.java:1073)
> > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
> > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
> > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1173)
> > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1090)
> > at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1110)
> > ..internal method calling getById()..
> > at java.lang.Thread.run(Thread.java:748)
> > Caused by: java.lang.IllegalStateException: Connection pool shut down
> > at org.apache.http.util.Asserts.check(Asserts.java:34)
> > at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
> > at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
> > at org.apache.http.impl.conn.PoolingClientConnectionManager
> > .requestConnection(PoolingClientConnectionManager.java:184)
> > at org.apache.http.impl.client.DefaultRequestDirector.execute(
> > DefaultRequestDirector.java:415)
> > at org.apache.http.impl.client.AbstractHttpClient.doExecute(
> > AbstractHttpClient.java:882)
> > at org.apache.http.impl.client.CloseableHttpClient.execute(
> > CloseableHttpClient.java:82)
> > at org.apache.http.impl.client.CloseableHttpClient.execute(
> > CloseableHttpClient.java:55)
> > at org.apache.solr.client.solrj.impl.HttpSolrClient.
> > executeMethod(HttpSolrClient.java:515)
> > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> > HttpSolrClient.java:279)
> > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> > HttpSolrClient.java:268)
> > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> > doRequest(LBHttpSolrClient.java:447)
> > ... 24 more
> >
> > So, to summarize, we have a program checking presence of documents in Solr
> > using getById() and we don't want this exception to bubble up, we want
> > SolrJ to restore the connection pool just as CloudSolrClient would move on
> > to another node if one went down in the mean time.
> >
> > Is this possible? How?
> >
> > Many thanks,
> > Markus
> >
> > -Original message-
> > > From:Markus Jelsma 
> > > Sent: Thursday 29th June 2017 16:38
> > > To: solr-user@lucene.apache.org
> > > Subject: RE: SolrJ 6.6.0 Connection pool shutdown
> > >
> > > Thanks. I probably should have mentioned there is no firewall limiting
> > connections between those hosts. Actually, the processes run on the same
> > hosts as the Solr cluster is running on.
> > >
> > > Thanks,
> > > Markus
> > >
> > >
> > >
> > > -Original message-
> > > > From:Alexandre Rafalovitch 
> > > > Sent: Thursday 29th June 2017 15:38
> > > > To: solr-user 
> > > > Subject: Re: SolrJ 6.6.0 Connection pool shutdown
> > > >
> > > > One thing to check is whether there is a firewall between the client
> > > > and the server. They - sometimes - cut the silent connections in the
> > > > _middle_ (at the firewall). The usual solution is keepAlive request of
> > > > some kind or not using the connection pool.
> > > >
> > > > One way to check is with network tracer like Wireshark and checking
> > > > whether the actual hardware at the other end of the connection is a
> > > > normal server or s

Re: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-07-18 Thread Susheel Kumar
Do you see any errors etc. in solr.log during this time?

On Tue, Jul 18, 2017 at 7:10 AM, Markus Jelsma 
wrote:

> The problem was never resolved but Shawn asked for the stack trace, here
> it is:
>
> org.apache.solr.client.solrj.SolrServerException: 
> java.lang.IllegalStateException:
> Connection pool shut down
> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> doRequest(LBHttpSolrClient.java:485)
> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(
> LBHttpSolrClient.java:388)
> at org.apache.solr.client.solrj.impl.CloudSolrClient.
> sendRequest(CloudSolrClient.java:1383)
> at org.apache.solr.client.solrj.impl.CloudSolrClient.
> requestWithRetryOnStaleState(CloudSolrClient.java:1134)
> at org.apache.solr.client.solrj.impl.CloudSolrClient.request(
> CloudSolrClient.java:1073)
> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
> at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
> at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1173)
> at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1090)
> at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1110)
> ..internal method calling getById()..
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: Connection pool shut down
> at org.apache.http.util.Asserts.check(Asserts.java:34)
> at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
> at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
> at org.apache.http.impl.conn.PoolingClientConnectionManager
> .requestConnection(PoolingClientConnectionManager.java:184)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(
> DefaultRequestDirector.java:415)
> at org.apache.http.impl.client.AbstractHttpClient.doExecute(
> AbstractHttpClient.java:882)
> at org.apache.http.impl.client.CloseableHttpClient.execute(
> CloseableHttpClient.java:82)
> at org.apache.http.impl.client.CloseableHttpClient.execute(
> CloseableHttpClient.java:55)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.
> executeMethod(HttpSolrClient.java:515)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> HttpSolrClient.java:279)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> HttpSolrClient.java:268)
> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
> doRequest(LBHttpSolrClient.java:447)
> ... 24 more
>
> So, to summarize, we have a program checking presence of documents in Solr
> using getById() and we don't want this exception to bubble up, we want
> SolrJ to restore the connection pool just as CloudSolrClient would move on
> to another node if one went down in the mean time.
>
> Is this possible? How?
>
> Many thanks,
> Markus
>
> -Original message-
> > From:Markus Jelsma 
> > Sent: Thursday 29th June 2017 16:38
> > To: solr-user@lucene.apache.org
> > Subject: RE: SolrJ 6.6.0 Connection pool shutdown
> >
> > Thanks. I probably should have mentioned there is no firewall limiting
> connections between those hosts. Actually, the processes run on the same
> hosts as the Solr cluster is running on.
> >
> > Thanks,
> > Markus
> >
> >
> >
> > -Original message-
> > > From:Alexandre Rafalovitch 
> > > Sent: Thursday 29th June 2017 15:38
> > > To: solr-user 
> > > Subject: Re: SolrJ 6.6.0 Connection pool shutdown
> > >
> > > One thing to check is whether there is a firewall between the client
> > > and the server. They - sometimes - cut the silent connections in the
> > > _middle_ (at the firewall). The usual solution is keepAlive request of
> > > some kind or not using the connection pool.
> > >
> > > One way to check is with network tracer like Wireshark and checking
> > > whether the actual hardware at the other end of the connection is a
> > > normal server or some sort of unexpected hardware piece of equipment
> > > (firewall). Yes, that's using the hammer to swat a fly :-)
> > >
> > > Regards,
> > >Alex.
> > > 
> > > http://www.solr-start.com/ - Resources for Solr users, new and
> experienced
> > >
> > >
> > > On 29 June 2017 at 08:21, Markus Jelsma 
> wrote:
> > > > Hi,
> > > >
> > > > Everything is 6.6.0. I could include a stack trace (i don't print
> them in my program), but that would only be the the trace from getById() to
> CloudSolrClient.requestWithRetryOnStaleState() and little deeper, that
> what you're looking for?
> > > >
> &

RE: SolrJ 6.6.0 Connection pool shutdown now with stack trace

2017-07-18 Thread Markus Jelsma
The problem was never resolved but Shawn asked for the stack trace, here it is:

org.apache.solr.client.solrj.SolrServerException: 
java.lang.IllegalStateException: Connection pool shut down 
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:485)
 
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
 
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
 
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
 
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
 
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160) 
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942) 
at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1173) 
at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1090) 
at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1110)
    ..internal method calling getById().. 
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Connection pool shut down 
at org.apache.http.util.Asserts.check(Asserts.java:34) 
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184) 
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217) 
at 
org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
 
at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
 
at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
 
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
 
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
 
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
 
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
 
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
 
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
 
... 24 more

So, to summarize, we have a program checking presence of documents in Solr 
using getById() and we don't want this exception to bubble up, we want SolrJ to 
restore the connection pool just as CloudSolrClient would move on to another 
node if one went down in the mean time.

Is this possible? How?

Many thanks,
Markus
 
-Original message-
> From:Markus Jelsma 
> Sent: Thursday 29th June 2017 16:38
> To: solr-user@lucene.apache.org
> Subject: RE: SolrJ 6.6.0 Connection pool shutdown
> 
> Thanks. I probably should have mentioned there is no firewall limiting 
> connections between those hosts. Actually, the processes run on the same 
> hosts as the Solr cluster is running on.
> 
> Thanks,
> Markus
> 
>  
>  
> -Original message-
> > From:Alexandre Rafalovitch 
> > Sent: Thursday 29th June 2017 15:38
> > To: solr-user 
> > Subject: Re: SolrJ 6.6.0 Connection pool shutdown
> > 
> > One thing to check is whether there is a firewall between the client
> > and the server. They - sometimes - cut the silent connections in the
> > _middle_ (at the firewall). The usual solution is keepAlive request of
> > some kind or not using the connection pool.
> > 
> > One way to check is with network tracer like Wireshark and checking
> > whether the actual hardware at the other end of the connection is a
> > normal server or some sort of unexpected hardware piece of equipment
> > (firewall). Yes, that's using the hammer to swat a fly :-)
> > 
> > Regards,
> >    Alex.
> > 
> > http://www.solr-start.com/ - Resources for Solr users, new and experienced
> > 
> > 
> > On 29 June 2017 at 08:21, Markus Jelsma  wrote:
> > > Hi,
> > >
> > > Everything is 6.6.0. I could include a stack trace (i don't print them in 
> > > my program), but that would only be the the trace from getById() to 
> > > CloudSolrClient.requestWithRetryOnStaleState() and little deeper, that 
> > > what you're looking for?
> > >
> > > We haven't called close() in that particular part of the program.
> > >
> > > Method requestWithRetryOnStaleState has some retry logic built-in but 
> > > doesn't seem to work for the exception i got.
> > >
> > > I'll let it print the stack trace and get back if it happens again.
> > >
> > > Thanks,
> > > Markus
> > >
> > > -Original message-
> > >> From:Shawn Heisey 
> 

RE: SolrJ 6.6.0 Connection pool shutdown

2017-06-29 Thread Markus Jelsma
Thanks. I probably should have mentioned there is no firewall limiting 
connections between those hosts. Actually, the processes run on the same hosts 
as the Solr cluster is running on.

Thanks,
Markus

 
 
-Original message-
> From:Alexandre Rafalovitch 
> Sent: Thursday 29th June 2017 15:38
> To: solr-user 
> Subject: Re: SolrJ 6.6.0 Connection pool shutdown
> 
> One thing to check is whether there is a firewall between the client
> and the server. They - sometimes - cut the silent connections in the
> _middle_ (at the firewall). The usual solution is keepAlive request of
> some kind or not using the connection pool.
> 
> One way to check is with network tracer like Wireshark and checking
> whether the actual hardware at the other end of the connection is a
> normal server or some sort of unexpected hardware piece of equipment
> (firewall). Yes, that's using the hammer to swat a fly :-)
> 
> Regards,
>Alex.
> 
> http://www.solr-start.com/ - Resources for Solr users, new and experienced
> 
> 
> On 29 June 2017 at 08:21, Markus Jelsma  wrote:
> > Hi,
> >
> > Everything is 6.6.0. I could include a stack trace (i don't print them in 
> > my program), but that would only be the the trace from getById() to 
> > CloudSolrClient.requestWithRetryOnStaleState() and little deeper, that what 
> > you're looking for?
> >
> > We haven't called close() in that particular part of the program.
> >
> > Method requestWithRetryOnStaleState has some retry logic built-in but 
> > doesn't seem to work for the exception i got.
> >
> > I'll let it print the stack trace and get back if it happens again.
> >
> > Thanks,
> > Markus
> >
> > -Original message-
> >> From:Shawn Heisey 
> >> Sent: Tuesday 27th June 2017 23:02
> >> To: solr-user@lucene.apache.org
> >> Subject: Re: SolrJ 6.6.0 Connection pool shutdown
> >>
> >> On 6/27/2017 6:50 AM, Markus Jelsma wrote:
> >> > We have a proces checking presence of many documents in a collection, 
> >> > just a simple client.getById(id). It sometimes begins throwing lots of  
> >> > these exceptions in a row:
> >> >
> >> > org.apache.solr.client.solrj.SolrServerException: 
> >> > java.lang.IllegalStateException: Connection pool shut down
> >> >
> >> > Then, as suddenly as it appeared, it's gone again a no longer a problem. 
> >> > I would expect SolrJ not to throw this but to wait until it the 
> >> > connection pool, or whatever mechanism is there, to recover.
> >> >
> >> > Did i miss a magic parameter for SolrJ?\
> >>
> >> That error message will be much longer than what you've provided here.
> >> It will have a java stacktrace that's typically a dozen or so lines
> >> long.  There may also be one or more "Caused by" sections after the
> >> stacktrace, each with a stacktrace of its own.  Can you share the full
> >> error message?  Is the server also running 6.6.0, or a different version?
> >>
> >> It would also be helpful if you can share the SolrJ code you've written,
> >> cleanly redacted to remove anything sensitive.
> >>
> >> That particular message ("Connection pool shut down") sounds like it
> >> probably came from HttpClient, which SolrJ uses ... and I would expect
> >> that to only happen if you close/shutdown the HttpClient or the
> >> SolrClient.  After closing, a client can't be used any more.  Normally
> >> the only time you should close a client is right before exiting the
> >> program, although if the program's about to exit, it's generally
> >> unnecessary, so in my opinion for *most* usages, closing the client
> >> likely never needs to happen.
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
> 


Re: SolrJ 6.6.0 Connection pool shutdown

2017-06-29 Thread Alexandre Rafalovitch
One thing to check is whether there is a firewall between the client
and the server. They - sometimes - cut the silent connections in the
_middle_ (at the firewall). The usual solution is keepAlive request of
some kind or not using the connection pool.

One way to check is with network tracer like Wireshark and checking
whether the actual hardware at the other end of the connection is a
normal server or some sort of unexpected hardware piece of equipment
(firewall). Yes, that's using the hammer to swat a fly :-)

Regards,
   Alex.

http://www.solr-start.com/ - Resources for Solr users, new and experienced


On 29 June 2017 at 08:21, Markus Jelsma  wrote:
> Hi,
>
> Everything is 6.6.0. I could include a stack trace (i don't print them in my 
> program), but that would only be the the trace from getById() to 
> CloudSolrClient.requestWithRetryOnStaleState() and little deeper, that what 
> you're looking for?
>
> We haven't called close() in that particular part of the program.
>
> Method requestWithRetryOnStaleState has some retry logic built-in but doesn't 
> seem to work for the exception i got.
>
> I'll let it print the stack trace and get back if it happens again.
>
> Thanks,
> Markus
>
> -Original message-
>> From:Shawn Heisey 
>> Sent: Tuesday 27th June 2017 23:02
>> To: solr-user@lucene.apache.org
>> Subject: Re: SolrJ 6.6.0 Connection pool shutdown
>>
>> On 6/27/2017 6:50 AM, Markus Jelsma wrote:
>> > We have a proces checking presence of many documents in a collection, just 
>> > a simple client.getById(id). It sometimes begins throwing lots of  these 
>> > exceptions in a row:
>> >
>> > org.apache.solr.client.solrj.SolrServerException: 
>> > java.lang.IllegalStateException: Connection pool shut down
>> >
>> > Then, as suddenly as it appeared, it's gone again a no longer a problem. I 
>> > would expect SolrJ not to throw this but to wait until it the connection 
>> > pool, or whatever mechanism is there, to recover.
>> >
>> > Did i miss a magic parameter for SolrJ?\
>>
>> That error message will be much longer than what you've provided here.
>> It will have a java stacktrace that's typically a dozen or so lines
>> long.  There may also be one or more "Caused by" sections after the
>> stacktrace, each with a stacktrace of its own.  Can you share the full
>> error message?  Is the server also running 6.6.0, or a different version?
>>
>> It would also be helpful if you can share the SolrJ code you've written,
>> cleanly redacted to remove anything sensitive.
>>
>> That particular message ("Connection pool shut down") sounds like it
>> probably came from HttpClient, which SolrJ uses ... and I would expect
>> that to only happen if you close/shutdown the HttpClient or the
>> SolrClient.  After closing, a client can't be used any more.  Normally
>> the only time you should close a client is right before exiting the
>> program, although if the program's about to exit, it's generally
>> unnecessary, so in my opinion for *most* usages, closing the client
>> likely never needs to happen.
>>
>> Thanks,
>> Shawn
>>
>>


RE: SolrJ 6.6.0 Connection pool shutdown

2017-06-29 Thread Markus Jelsma
Hi,

Everything is 6.6.0. I could include a stack trace (i don't print them in my 
program), but that would only be the the trace from getById() to 
CloudSolrClient.requestWithRetryOnStaleState() and little deeper, that what 
you're looking for?

We haven't called close() in that particular part of the program.

Method requestWithRetryOnStaleState has some retry logic built-in but doesn't 
seem to work for the exception i got. 

I'll let it print the stack trace and get back if it happens again.

Thanks,
Markus

-Original message-
> From:Shawn Heisey 
> Sent: Tuesday 27th June 2017 23:02
> To: solr-user@lucene.apache.org
> Subject: Re: SolrJ 6.6.0 Connection pool shutdown
> 
> On 6/27/2017 6:50 AM, Markus Jelsma wrote:
> > We have a proces checking presence of many documents in a collection, just 
> > a simple client.getById(id). It sometimes begins throwing lots of  these 
> > exceptions in a row:
> >
> > org.apache.solr.client.solrj.SolrServerException: 
> > java.lang.IllegalStateException: Connection pool shut down
> >
> > Then, as suddenly as it appeared, it's gone again a no longer a problem. I 
> > would expect SolrJ not to throw this but to wait until it the connection 
> > pool, or whatever mechanism is there, to recover.
> >
> > Did i miss a magic parameter for SolrJ?\
> 
> That error message will be much longer than what you've provided here. 
> It will have a java stacktrace that's typically a dozen or so lines
> long.  There may also be one or more "Caused by" sections after the
> stacktrace, each with a stacktrace of its own.  Can you share the full
> error message?  Is the server also running 6.6.0, or a different version?
> 
> It would also be helpful if you can share the SolrJ code you've written,
> cleanly redacted to remove anything sensitive.
> 
> That particular message ("Connection pool shut down") sounds like it
> probably came from HttpClient, which SolrJ uses ... and I would expect
> that to only happen if you close/shutdown the HttpClient or the
> SolrClient.  After closing, a client can't be used any more.  Normally
> the only time you should close a client is right before exiting the
> program, although if the program's about to exit, it's generally
> unnecessary, so in my opinion for *most* usages, closing the client
> likely never needs to happen.
> 
> Thanks,
> Shawn
> 
> 


Re: SolrJ 6.6.0 Connection pool shutdown

2017-06-27 Thread Shawn Heisey
On 6/27/2017 6:50 AM, Markus Jelsma wrote:
> We have a proces checking presence of many documents in a collection, just a 
> simple client.getById(id). It sometimes begins throwing lots of  these 
> exceptions in a row:
>
> org.apache.solr.client.solrj.SolrServerException: 
> java.lang.IllegalStateException: Connection pool shut down
>
> Then, as suddenly as it appeared, it's gone again a no longer a problem. I 
> would expect SolrJ not to throw this but to wait until it the connection 
> pool, or whatever mechanism is there, to recover.
>
> Did i miss a magic parameter for SolrJ?\

That error message will be much longer than what you've provided here. 
It will have a java stacktrace that's typically a dozen or so lines
long.  There may also be one or more "Caused by" sections after the
stacktrace, each with a stacktrace of its own.  Can you share the full
error message?  Is the server also running 6.6.0, or a different version?

It would also be helpful if you can share the SolrJ code you've written,
cleanly redacted to remove anything sensitive.

That particular message ("Connection pool shut down") sounds like it
probably came from HttpClient, which SolrJ uses ... and I would expect
that to only happen if you close/shutdown the HttpClient or the
SolrClient.  After closing, a client can't be used any more.  Normally
the only time you should close a client is right before exiting the
program, although if the program's about to exit, it's generally
unnecessary, so in my opinion for *most* usages, closing the client
likely never needs to happen.

Thanks,
Shawn



SolrJ 6.6.0 Connection pool shutdown

2017-06-27 Thread Markus Jelsma
Hi,

We have a proces checking presence of many documents in a collection, just a 
simple client.getById(id). It sometimes begins throwing lots of  these 
exceptions in a row:

org.apache.solr.client.solrj.SolrServerException: 
java.lang.IllegalStateException: Connection pool shut down

Then, as suddenly as it appeared, it's gone again a no longer a problem. I 
would expect SolrJ not to throw this but to wait until it the connection pool, 
or whatever mechanism is there, to recover.

Did i miss a magic parameter for SolrJ?

Thanks,
Markus


Re: Connection pool shutdown error

2015-03-19 Thread Andrea Gazzarini
I bet the problem is how the SolrServer instance is used within Spring 
Repository. I think somewhere you should alternatively


- explicitly close the client each time.
- reuse the same instance (and finally close that)

But being a Spring newbie I cannot give you further information.

Best,
Andrea

On 03/19/2015 02:18 PM, phi...@free.fr wrote:

Hello,

I am trying to use the 4.9.1 SOLR Core API and the 1.3.2.RELEASE version of the 
Spring Data SOLR API, to connect to a SOLR server, but to no avail.

When I run Java application, I get the following errors:

---

Exception in thread "main" 
org.springframework.data.solr.UncategorizedSolrException: Error executing query; nested 
exception is org.apache.solr.client.solrj.SolrServerException: Error executing query
...
Caused by: java.lang.IllegalStateException: Connection pool shut down

-

I have tried changing Core API version (4.3.0, 4.4.0, ...) but to no avail.

Any help would be much appreciated.

Cheers,

Philippe




Here's my Solr Context:



package com.myco.archives.SolrGuiMain;



@Configuration
@EnableSolrRepositories(basePackages = { "com.myco.archives" }, 
multicoreSupport = false)
@ComponentScan
public class SolrContext {

private final StringHTTP_SEARCHARCHIVES = 
"http://mysolr.com:8990/solr/collection3";;

@Bean
public SolrServer solrServer() {
SolrServer server = new HttpSolrServer(HTTP_SEARCHARCHIVES);
return server;
}

@Bean
public SolrOperations solrTemplate() {
return new SolrTemplate(solrServer());
}

}

-

Here's my Repository Class:

import org.springframework.data.repository.CrudRepository;

public interface ArchiveDocumentRepository extends CrudRepository {

List findByText(String text);

List findByYmd(Date ymd);

}




And here's my App:

import 
org.springframework.context.annotation.AnnotationConfigApplicationContext;

public class App
{

private ArchiveDocumentRepository   archiveDocumentRepository;

public App() {

setContext();
processDocs();


}
public static void main(String[] args) {

new App();

}

public void setContext() throws RuntimeException {

AnnotationConfigApplicationContext context = new 
AnnotationConfigApplicationContext(SolrContext.class);

if (context != null) {

setArchiveDocumentRepository(context.getBean(ArchiveDocumentRepository.class));
}
context.close();
}

public final ArchiveDocumentRepository getArchiveDocumentRepository() {
return archiveDocumentRepository;
}

public final void 
setArchiveDocumentRepository(ArchiveDocumentRepository 
archiveDocumentRepository) {
this.archiveDocumentRepository = archiveDocumentRepository;
}

public void processDocs() {


Iterable docs = 
getArchiveDocumentRepository().findAll();

for (Document doc : docs) {
System.out.println("doc count = " + doc.getYmd());
}

}

}


---












Connection pool shutdown error

2015-03-19 Thread phiroc
Hello,

I am trying to use the 4.9.1 SOLR Core API and the 1.3.2.RELEASE version of the 
Spring Data SOLR API, to connect to a SOLR server, but to no avail.

When I run Java application, I get the following errors:

---

Exception in thread "main" 
org.springframework.data.solr.UncategorizedSolrException: Error executing 
query; nested exception is org.apache.solr.client.solrj.SolrServerException: 
Error executing query
...
Caused by: java.lang.IllegalStateException: Connection pool shut down

-

I have tried changing Core API version (4.3.0, 4.4.0, ...) but to no avail.

Any help would be much appreciated.

Cheers,

Philippe




Here's my Solr Context:



package com.myco.archives.SolrGuiMain;



@Configuration
@EnableSolrRepositories(basePackages = { "com.myco.archives" }, 
multicoreSupport = false)
@ComponentScan
public class SolrContext {

private final StringHTTP_SEARCHARCHIVES = 
"http://mysolr.com:8990/solr/collection3";;

@Bean
public SolrServer solrServer() {
SolrServer server = new HttpSolrServer(HTTP_SEARCHARCHIVES);
return server;
}

@Bean
public SolrOperations solrTemplate() {
return new SolrTemplate(solrServer());
}

}

-

Here's my Repository Class:

import org.springframework.data.repository.CrudRepository;

public interface ArchiveDocumentRepository extends CrudRepository {

List findByText(String text);

List findByYmd(Date ymd);

}




And here's my App:

import 
org.springframework.context.annotation.AnnotationConfigApplicationContext;

public class App
{

private ArchiveDocumentRepository   archiveDocumentRepository;

public App() {

setContext();
processDocs();


}
public static void main(String[] args) {

new App();

}

public void setContext() throws RuntimeException {

AnnotationConfigApplicationContext context = new 
AnnotationConfigApplicationContext(SolrContext.class);

if (context != null) {

setArchiveDocumentRepository(context.getBean(ArchiveDocumentRepository.class));
}
context.close();
}

public final ArchiveDocumentRepository getArchiveDocumentRepository() {
return archiveDocumentRepository;
}

public final void 
setArchiveDocumentRepository(ArchiveDocumentRepository 
archiveDocumentRepository) {
this.archiveDocumentRepository = archiveDocumentRepository;
}

public void processDocs() {


Iterable docs = 
getArchiveDocumentRepository().findAll();

for (Document doc : docs) {
System.out.println("doc count = " + doc.getYmd());
}

}

}


---










Re: DIH using a connection pool

2012-05-24 Thread Lance Norskog
Yes, this is the right way for the DIH. You might find it easier to
write a separate local client that polls the DB and uploads changes.
The DIH is oriented toward longer batch jobs.

On Thu, May 24, 2012 at 7:29 AM, Esteban Donato
 wrote:
> Hi community,
>
> I am using Solr with DIH to index content from a DB.  The point is
> that I have to configure DIH to check changes in the DB very
> frequently (aprox 1 sec) to maintain the index almost up-to-date.  I
> noted that JDBCDataSource closes the DB connection after every
> execution which is not acceptable with this update rate.  Ideally I
> would need DIH using a connection pool.  Looking at DIH code and faq I
> noticed that I can configure a connection pool and expose it via jndi
> for the JDBCDataSource to use it.  My question is: is this they way to
> go for integrating a connection pool with DIH?
>
> Thanks
> Esteban



-- 
Lance Norskog
goks...@gmail.com


DIH using a connection pool

2012-05-24 Thread Esteban Donato
Hi community,

I am using Solr with DIH to index content from a DB.  The point is
that I have to configure DIH to check changes in the DB very
frequently (aprox 1 sec) to maintain the index almost up-to-date.  I
noted that JDBCDataSource closes the DB connection after every
execution which is not acceptable with this update rate.  Ideally I
would need DIH using a connection pool.  Looking at DIH code and faq I
noticed that I can configure a connection pool and expose it via jndi
for the JDBCDataSource to use it.  My question is: is this they way to
go for integrating a connection pool with DIH?

Thanks
Esteban


Re: Connection Pool

2010-05-18 Thread Lance Norskog
Do multiple calls with your client program. So,
curl _file1_ &
curl _file2_ &
curl _file3_ &
curl _file4_ &
wait; wait; wait; wait


On Sun, May 16, 2010 at 8:20 AM, Monmohan Singh  wrote:
> Sorry for hijacking the thread, but I have an additional question
> Is there a way to achieve similar performance (SUSS like) when targeting
> extract request handler (/update/extract)?
> I guess one way can be to extract content on the client side and then use
> SUSS to send update request but then extraction needs to be taken care of
> locally in an asynchronous/batch manner.
> Regards
> Monmohan
>
> On Sun, May 16, 2010 at 5:19 AM, Lance Norskog  wrote:
>
>> Connection spooling is specified by the underlying apache commons
>> connection manager when you create the Server.
>>
>> The SUSS does socket pooling by default and is the preferred way to do
>> concurrent indexing. There are some quirks in the Server
>> implementation set, and SUSS avoids them. Unless you are willing to
>> root around in the SolrJ Server code and understand exactly how it
>> works, stay with the SUSS.
>>
>> On Fri, May 14, 2010 at 6:44 AM, gabriele renzi  wrote:
>> > On Fri, May 14, 2010 at 3:35 PM, Anderson vasconcelos
>> >  wrote:
>> >> Hi
>> >> I wanna to know if has any connection pool client to manage the
>> connections
>> >> with solr. In my system, we have a lot of concurrency index request. I
>> cant
>> >> shared my  connection, i need to create one per transaction. But if i
>> create
>> >> one per transaction, i think the performance will down.
>> >>
>> >> How you resolve this problem?
>> >
>> > The commonsHttpSolrServer class does connection pooling, and IIRC also
>> > the StreamingUpdateSolrServer.
>> >
>> >
>> >
>> > --
>> > blog en: http://www.riffraff.info
>> > blog it: http://riffraff.blogsome.com
>> >
>>
>>
>>
>> --
>> Lance Norskog
>> goks...@gmail.com
>>
>



-- 
Lance Norskog
goks...@gmail.com


Re: Connection Pool

2010-05-16 Thread Monmohan Singh
Sorry for hijacking the thread, but I have an additional question
Is there a way to achieve similar performance (SUSS like) when targeting
extract request handler (/update/extract)?
I guess one way can be to extract content on the client side and then use
SUSS to send update request but then extraction needs to be taken care of
locally in an asynchronous/batch manner.
Regards
Monmohan

On Sun, May 16, 2010 at 5:19 AM, Lance Norskog  wrote:

> Connection spooling is specified by the underlying apache commons
> connection manager when you create the Server.
>
> The SUSS does socket pooling by default and is the preferred way to do
> concurrent indexing. There are some quirks in the Server
> implementation set, and SUSS avoids them. Unless you are willing to
> root around in the SolrJ Server code and understand exactly how it
> works, stay with the SUSS.
>
> On Fri, May 14, 2010 at 6:44 AM, gabriele renzi  wrote:
> > On Fri, May 14, 2010 at 3:35 PM, Anderson vasconcelos
> >  wrote:
> >> Hi
> >> I wanna to know if has any connection pool client to manage the
> connections
> >> with solr. In my system, we have a lot of concurrency index request. I
> cant
> >> shared my  connection, i need to create one per transaction. But if i
> create
> >> one per transaction, i think the performance will down.
> >>
> >> How you resolve this problem?
> >
> > The commonsHttpSolrServer class does connection pooling, and IIRC also
> > the StreamingUpdateSolrServer.
> >
> >
> >
> > --
> > blog en: http://www.riffraff.info
> > blog it: http://riffraff.blogsome.com
> >
>
>
>
> --
> Lance Norskog
> goks...@gmail.com
>


Re: Connection Pool

2010-05-15 Thread Lance Norskog
Connection spooling is specified by the underlying apache commons
connection manager when you create the Server.

The SUSS does socket pooling by default and is the preferred way to do
concurrent indexing. There are some quirks in the Server
implementation set, and SUSS avoids them. Unless you are willing to
root around in the SolrJ Server code and understand exactly how it
works, stay with the SUSS.

On Fri, May 14, 2010 at 6:44 AM, gabriele renzi  wrote:
> On Fri, May 14, 2010 at 3:35 PM, Anderson vasconcelos
>  wrote:
>> Hi
>> I wanna to know if has any connection pool client to manage the connections
>> with solr. In my system, we have a lot of concurrency index request. I cant
>> shared my  connection, i need to create one per transaction. But if i create
>> one per transaction, i think the performance will down.
>>
>> How you resolve this problem?
>
> The commonsHttpSolrServer class does connection pooling, and IIRC also
> the StreamingUpdateSolrServer.
>
>
>
> --
> blog en: http://www.riffraff.info
> blog it: http://riffraff.blogsome.com
>



-- 
Lance Norskog
goks...@gmail.com


Re: Connection Pool

2010-05-14 Thread gabriele renzi
On Fri, May 14, 2010 at 3:35 PM, Anderson vasconcelos
 wrote:
> Hi
> I wanna to know if has any connection pool client to manage the connections
> with solr. In my system, we have a lot of concurrency index request. I cant
> shared my  connection, i need to create one per transaction. But if i create
> one per transaction, i think the performance will down.
>
> How you resolve this problem?

The commonsHttpSolrServer class does connection pooling, and IIRC also
the StreamingUpdateSolrServer.



-- 
blog en: http://www.riffraff.info
blog it: http://riffraff.blogsome.com


Connection Pool

2010-05-14 Thread Anderson vasconcelos
Hi
I wanna to know if has any connection pool client to manage the connections
with solr. In my system, we have a lot of concurrency index request. I cant
shared my  connection, i need to create one per transaction. But if i create
one per transaction, i think the performance will down.

How you resolve this problem?

Thanks