Re: Multiple concurrent queries to Solr

2015-08-24 Thread Ashish Mukherjee
Thanks, everyone. Arcadius, that ticket is interesting.

I was wondering if an implementation of SolrClient could be based on
HttpAsyncClient
instead of HttpSolrClient. Just a thought right now, which needs to be
explored deeper.

- Ashish

On Mon, Aug 24, 2015 at 1:46 AM, Arcadius Ahouansou arcad...@menelic.com
wrote:

 Hello Ashish.

 Therse is an unfinished work about this at
 https://issues.apache.org/jira/browse/SOLR-3383

 Maybe you want to have a look and contribute?

 Arcadius.

 On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org
 wrote:

  The last time that I used the HTTPClient library, it was non-blocking. It
  doesn’t try to read from the socket until you ask for data from the
  response object. That allows parallel requests without threads.
 
  Underneath, it has a pool of connections that can be reused. If the pool
  is exhausted, it can block.
 
  wunder
  Walter Underwood
  wun...@wunderwood.org
  http://observer.wunderwood.org/  (my blog)
 
 
  On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote:
 
   On 8/23/2015 7:46 AM, Ashish Mukherjee wrote:
   I want to run few Solr queries in parallel, which are being done in a
   multi-threaded model now. I was wondering if there are any client
  libraries
   to query Solr  through a non-blocking I/O mechanism instead of a
  threaded
   model. Has anyone attempted something like this?
  
   The only client library that the Solr project makes is SolrJ -- the
   client for Java.  If you are not using the SolrJ client, then the Solr
   project did not write it, and you should contact the authors of the
   library directly.
  
   SolrJ and Solr are both completely thread-safe, and multiple threads
 are
   recommended for highly concurrent usage.  SolrJ uses HttpClient for
   communication with Solr.
  
   I was not able to determine whether the default httpclient settings
 will
   result in non-blocking I/O or not. As far as I am aware, nothing in
   SolrJ sets any explicit configuration for blocking or non-blocking I/O.
   You can create your own HttpClient object in a SolrJ program and have
   the SolrClient object use it.
  
   HttpClient uses HttpCore.  Here is the main web page for these
  components:
  
   https://hc.apache.org/
  
   On this webpage, it says HttpCore supports two I/O models: blocking
 I/O
   model based on the classic Java I/O and non-blocking, event driven I/O
   model based on Java NIO.  There is no information here about which
   model is chosen by default.
  
   Thanks,
   Shawn
  
 
 


 --
 Arcadius Ahouansou
 Menelic Ltd | Information is Power
 M: 07908761999
 W: www.menelic.com
 ---



Re: Multiple concurrent queries to Solr

2015-08-24 Thread Arcadius Ahouansou
BTW, google revealed that there is a 3rd-party Scala library for async
calls which could be usable from Java.
I have not tried it myself though
https://github.com/inoio/solrs


On 24 August 2015 at 21:35, Arcadius Ahouansou arcad...@menelic.com wrote:

 Hi Ashish.

 The Apache HttpAsyncClient uses Java Future to wrap a synchronous call
 into asyn
 The above ticket does similar thing by wrapping a SolrJ call into Future

 Feel free to submit any proposal you may have to the dev list.

 Arcadius

 On 24 August 2015 at 07:20, Ashish Mukherjee ashish.mukher...@gmail.com
 wrote:

 Thanks, everyone. Arcadius, that ticket is interesting.

 I was wondering if an implementation of SolrClient could be based on
 HttpAsyncClient
 instead of HttpSolrClient. Just a thought right now, which needs to be
 explored deeper.

 - Ashish

 On Mon, Aug 24, 2015 at 1:46 AM, Arcadius Ahouansou arcad...@menelic.com
 
 wrote:

  Hello Ashish.
 
  Therse is an unfinished work about this at
  https://issues.apache.org/jira/browse/SOLR-3383
 
  Maybe you want to have a look and contribute?
 
  Arcadius.
 
  On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org
  wrote:
 
   The last time that I used the HTTPClient library, it was
 non-blocking. It
   doesn’t try to read from the socket until you ask for data from the
   response object. That allows parallel requests without threads.
  
   Underneath, it has a pool of connections that can be reused. If the
 pool
   is exhausted, it can block.
  
   wunder
   Walter Underwood
   wun...@wunderwood.org
   http://observer.wunderwood.org/  (my blog)
  
  
   On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org
 wrote:
  
On 8/23/2015 7:46 AM, Ashish Mukherjee wrote:
I want to run few Solr queries in parallel, which are being done
 in a
multi-threaded model now. I was wondering if there are any client
   libraries
to query Solr  through a non-blocking I/O mechanism instead of a
   threaded
model. Has anyone attempted something like this?
   
The only client library that the Solr project makes is SolrJ -- the
client for Java.  If you are not using the SolrJ client, then the
 Solr
project did not write it, and you should contact the authors of the
library directly.
   
SolrJ and Solr are both completely thread-safe, and multiple threads
  are
recommended for highly concurrent usage.  SolrJ uses HttpClient for
communication with Solr.
   
I was not able to determine whether the default httpclient settings
  will
result in non-blocking I/O or not. As far as I am aware, nothing in
SolrJ sets any explicit configuration for blocking or non-blocking
 I/O.
You can create your own HttpClient object in a SolrJ program and
 have
the SolrClient object use it.
   
HttpClient uses HttpCore.  Here is the main web page for these
   components:
   
https://hc.apache.org/
   
On this webpage, it says HttpCore supports two I/O models: blocking
  I/O
model based on the classic Java I/O and non-blocking, event driven
 I/O
model based on Java NIO.  There is no information here about which
model is chosen by default.
   
Thanks,
Shawn
   
  
  
 
 
  --
  Arcadius Ahouansou
  Menelic Ltd | Information is Power
  M: 07908761999
  W: www.menelic.com
  ---
 




 --
 Arcadius Ahouansou
 Menelic Ltd | Information is Power
 M: 07908761999
 W: www.menelic.com
 ---




-- 
Arcadius Ahouansou
Menelic Ltd | Information is Power
M: 07908761999
W: www.menelic.com
---


Re: Multiple concurrent queries to Solr

2015-08-24 Thread Arcadius Ahouansou
Hi Ashish.

The Apache HttpAsyncClient uses Java Future to wrap a synchronous call into
asyn
The above ticket does similar thing by wrapping a SolrJ call into Future

Feel free to submit any proposal you may have to the dev list.

Arcadius

On 24 August 2015 at 07:20, Ashish Mukherjee ashish.mukher...@gmail.com
wrote:

 Thanks, everyone. Arcadius, that ticket is interesting.

 I was wondering if an implementation of SolrClient could be based on
 HttpAsyncClient
 instead of HttpSolrClient. Just a thought right now, which needs to be
 explored deeper.

 - Ashish

 On Mon, Aug 24, 2015 at 1:46 AM, Arcadius Ahouansou arcad...@menelic.com
 wrote:

  Hello Ashish.
 
  Therse is an unfinished work about this at
  https://issues.apache.org/jira/browse/SOLR-3383
 
  Maybe you want to have a look and contribute?
 
  Arcadius.
 
  On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org
  wrote:
 
   The last time that I used the HTTPClient library, it was non-blocking.
 It
   doesn’t try to read from the socket until you ask for data from the
   response object. That allows parallel requests without threads.
  
   Underneath, it has a pool of connections that can be reused. If the
 pool
   is exhausted, it can block.
  
   wunder
   Walter Underwood
   wun...@wunderwood.org
   http://observer.wunderwood.org/  (my blog)
  
  
   On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote:
  
On 8/23/2015 7:46 AM, Ashish Mukherjee wrote:
I want to run few Solr queries in parallel, which are being done in
 a
multi-threaded model now. I was wondering if there are any client
   libraries
to query Solr  through a non-blocking I/O mechanism instead of a
   threaded
model. Has anyone attempted something like this?
   
The only client library that the Solr project makes is SolrJ -- the
client for Java.  If you are not using the SolrJ client, then the
 Solr
project did not write it, and you should contact the authors of the
library directly.
   
SolrJ and Solr are both completely thread-safe, and multiple threads
  are
recommended for highly concurrent usage.  SolrJ uses HttpClient for
communication with Solr.
   
I was not able to determine whether the default httpclient settings
  will
result in non-blocking I/O or not. As far as I am aware, nothing in
SolrJ sets any explicit configuration for blocking or non-blocking
 I/O.
You can create your own HttpClient object in a SolrJ program and have
the SolrClient object use it.
   
HttpClient uses HttpCore.  Here is the main web page for these
   components:
   
https://hc.apache.org/
   
On this webpage, it says HttpCore supports two I/O models: blocking
  I/O
model based on the classic Java I/O and non-blocking, event driven
 I/O
model based on Java NIO.  There is no information here about which
model is chosen by default.
   
Thanks,
Shawn
   
  
  
 
 
  --
  Arcadius Ahouansou
  Menelic Ltd | Information is Power
  M: 07908761999
  W: www.menelic.com
  ---
 




-- 
Arcadius Ahouansou
Menelic Ltd | Information is Power
M: 07908761999
W: www.menelic.com
---


Re: Multiple concurrent queries to Solr

2015-08-23 Thread Shawn Heisey
On 8/23/2015 7:46 AM, Ashish Mukherjee wrote:
 I want to run few Solr queries in parallel, which are being done in a
 multi-threaded model now. I was wondering if there are any client libraries
 to query Solr  through a non-blocking I/O mechanism instead of a threaded
 model. Has anyone attempted something like this?

The only client library that the Solr project makes is SolrJ -- the
client for Java.  If you are not using the SolrJ client, then the Solr
project did not write it, and you should contact the authors of the
library directly.

SolrJ and Solr are both completely thread-safe, and multiple threads are
recommended for highly concurrent usage.  SolrJ uses HttpClient for
communication with Solr.

I was not able to determine whether the default httpclient settings will
result in non-blocking I/O or not. As far as I am aware, nothing in
SolrJ sets any explicit configuration for blocking or non-blocking I/O.
 You can create your own HttpClient object in a SolrJ program and have
the SolrClient object use it.

HttpClient uses HttpCore.  Here is the main web page for these components:

https://hc.apache.org/

On this webpage, it says HttpCore supports two I/O models: blocking I/O
model based on the classic Java I/O and non-blocking, event driven I/O
model based on Java NIO.  There is no information here about which
model is chosen by default.

Thanks,
Shawn



Re: Multiple concurrent queries to Solr

2015-08-23 Thread Walter Underwood
The last time that I used the HTTPClient library, it was non-blocking. It 
doesn’t try to read from the socket until you ask for data from the response 
object. That allows parallel requests without threads.

Underneath, it has a pool of connections that can be reused. If the pool is 
exhausted, it can block.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote:

 On 8/23/2015 7:46 AM, Ashish Mukherjee wrote:
 I want to run few Solr queries in parallel, which are being done in a
 multi-threaded model now. I was wondering if there are any client libraries
 to query Solr  through a non-blocking I/O mechanism instead of a threaded
 model. Has anyone attempted something like this?
 
 The only client library that the Solr project makes is SolrJ -- the
 client for Java.  If you are not using the SolrJ client, then the Solr
 project did not write it, and you should contact the authors of the
 library directly.
 
 SolrJ and Solr are both completely thread-safe, and multiple threads are
 recommended for highly concurrent usage.  SolrJ uses HttpClient for
 communication with Solr.
 
 I was not able to determine whether the default httpclient settings will
 result in non-blocking I/O or not. As far as I am aware, nothing in
 SolrJ sets any explicit configuration for blocking or non-blocking I/O.
 You can create your own HttpClient object in a SolrJ program and have
 the SolrClient object use it.
 
 HttpClient uses HttpCore.  Here is the main web page for these components:
 
 https://hc.apache.org/
 
 On this webpage, it says HttpCore supports two I/O models: blocking I/O
 model based on the classic Java I/O and non-blocking, event driven I/O
 model based on Java NIO.  There is no information here about which
 model is chosen by default.
 
 Thanks,
 Shawn
 



Re: Multiple concurrent queries to Solr

2015-08-23 Thread Arcadius Ahouansou
Hello Ashish.

Therse is an unfinished work about this at
https://issues.apache.org/jira/browse/SOLR-3383

Maybe you want to have a look and contribute?

Arcadius.

On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org wrote:

 The last time that I used the HTTPClient library, it was non-blocking. It
 doesn’t try to read from the socket until you ask for data from the
 response object. That allows parallel requests without threads.

 Underneath, it has a pool of connections that can be reused. If the pool
 is exhausted, it can block.

 wunder
 Walter Underwood
 wun...@wunderwood.org
 http://observer.wunderwood.org/  (my blog)


 On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote:

  On 8/23/2015 7:46 AM, Ashish Mukherjee wrote:
  I want to run few Solr queries in parallel, which are being done in a
  multi-threaded model now. I was wondering if there are any client
 libraries
  to query Solr  through a non-blocking I/O mechanism instead of a
 threaded
  model. Has anyone attempted something like this?
 
  The only client library that the Solr project makes is SolrJ -- the
  client for Java.  If you are not using the SolrJ client, then the Solr
  project did not write it, and you should contact the authors of the
  library directly.
 
  SolrJ and Solr are both completely thread-safe, and multiple threads are
  recommended for highly concurrent usage.  SolrJ uses HttpClient for
  communication with Solr.
 
  I was not able to determine whether the default httpclient settings will
  result in non-blocking I/O or not. As far as I am aware, nothing in
  SolrJ sets any explicit configuration for blocking or non-blocking I/O.
  You can create your own HttpClient object in a SolrJ program and have
  the SolrClient object use it.
 
  HttpClient uses HttpCore.  Here is the main web page for these
 components:
 
  https://hc.apache.org/
 
  On this webpage, it says HttpCore supports two I/O models: blocking I/O
  model based on the classic Java I/O and non-blocking, event driven I/O
  model based on Java NIO.  There is no information here about which
  model is chosen by default.
 
  Thanks,
  Shawn
 




-- 
Arcadius Ahouansou
Menelic Ltd | Information is Power
M: 07908761999
W: www.menelic.com
---